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iiO _ Introduct i on 


The  evaluation  of  the  usefulness  of  color  in  the  Patriot 
high  altitude  air  defense  weapon  system  display  reported  here  is 
the  third  and  final  phase  of  research  performed  by  the  Army 
Research  Institute  (ARI)  Fort  Bl iss  Field  Uni t  for  the 
Directorate  of  Combat  Developments  (DCD),  US  Army  Air  Defense 
Art i I  1 ery  School  (USAADASCH) .  Phases  1  and  2  of  the  research 
were  (Da  survey  of  color  research,  and  (2)  the  development  of 
the  experimental  color  display.  These  two  phases  were  reported 
in  the  ARI  Working  Paper,  "Preliminary  Recommendations  on  the  Use 
of  Color  in  the  Patriot  Display"  (Al lender,  1986);  Phase  3 
consisted  of  an  experiment  run  on  the  Patriot  Tactical  Operations 
Simulator  (PTOS)  comparing  how  trained  Patriot  operators 
performed  using  an  experimental  color  display  with  how  they 
performed  using  the  existing  monochrome  display. 

The  entire  research  effort  was  prompted  by  two  computer 
technology  developments,  one,  a  contributor  to  the  problem,  and 
the  other,  a  possible  solution.  The  problem  is  that,  since 
computers  are  now  faster  and  capable  of  displaying  more 
information  than  ever  before,  the  information  processing  limits 
of  the  human  are  being  pushed  and  often  exceeded.  A  case  in 
point  is  the  Patriot,  which  is  described  as  the  Army's  first 
ful  I y  computerized  weapon  system  (FM.^fl-iSri) .  The  amount,  kind, 
and  timeliness  of  the  information  required  by  the  Patriot  is  in 
direct  response  to  the  threat,  which  comprises  more  targets 
flying  higher  and  faster  and  able  to  strike  from  farther  out  than 
before.  In  turn,  the  operator  is  presented  with  information  on  a 
rapidly  changing,  high  density  display,  and  is  required  to  make 

complex  decisions  based  on  that  information  in  a  short  period  of 
time. 

A  possible  solution  to  the  problem  of  computer  display 
clutter  and  information  overload  is  color  display  technology. 
Since  it  has  advanced  to  the  point  where  literally  millions  of 
colors  can  be  generated  and  displayed  in  high  resolution,  real 
time  graphics,  the  potential  for  color  to  enhance  information 
display  IS  increased.  Formerly,  the  use  of  color  was  limited  to 
coding  symbols  or  text  in  only  three  or  four  different  colors. 
Now,  with  greater  precision  in  color  selection  and  improved 
display  capabi I i ties,  the  use  of  color  can  be  extended  to  Include 
highlighting  critical  information  and  reducing  overall  display 
clutter.  This  research  effort,  then,  is  an  evaluation  of  the 
application  of  the  new  color  technology  to  the  problem  of 
information  display  overload. 

The  organization  of  this  report  is  as  follows;  Phases  1  and 
2  Of  this  effort,  the  survey  of  color  research  and  the 
development  of  the  experimental  color  display,  are  reviewed 
briefly  in  Section  2.0.  The  method  and  procedure  for  the 
evaluation  are  presented  in  Section  3.0  with  emphasis  on  the 
performance  measures  and  data  collection  techniques.  The  results 
of  the  experimental  evaluation  are  discussed  in  Section  4.0,  and 
the  conclusions  and  recommendations  are  presented  in  Section  5.0. 


2^0 _ Reyi ew_of _Phases  1  and  2 


_ Syrvei/_Qf _Cgior_Research .  Many  different  techniques 

have  been  shown  by  psychological  research  to  improve  the  display 
of  information.  A  number  of  them  are  already  employed  in  the 
design  of  the  Patriot  display — symbol,  shape  and  line  coding, 
brightness  coding,  blink-rate  coding,  and  formatting  of  tables. 
Color  can  also  improve  the  display  of  information  by  coding 
categories  of  information,  by  highlighting  critical  information, 
and  by  reducing  display  clutter.  However,  Just  the  addition  of 
color  to  a  display  does  not  ensure  an  improvement.  To  ensure 
that  color  is  used  to  advantage,  at  least  three  factors  must  be 
considered.  (For  a  more  detailed  discussion  of  the  use  of  color, 
see  A1 lender,  1986.)  For  one,  the  display  colors  must  be  chosen 
carefully  since  the  perception  of  a  foreground  object  depends  on 
the  color  of  the  surrounding  area  or  background,  particularly  if 
the  object  is  small.  Poor  foreground/background  contrast  makes 
an  object  hard  to  see.  Also,  certain  foreground/background 
combinations  can  even  change  the  perception  of  the  color  of  the 
foreground  object. 

A  second  factor  to  be  considered  in  using  color  is  that 
there  are  tradeoffs  and  limitations.  Color  cannot  code  all 
available  information;  It  cannot  code  both  background  and 
foreground  information  or  different  sections  of  a  display  with 
equal  priority.  Further,  it  cannot  code  more  than  one  category 
of  information  for  a  given  symbol  or  object.  For  example,  it 
cannot  be  used  effectively  to  code  both  the  identity  and  the 
altitude  of  an  aircraft.  Also,  the  number  of  different  color 
coded  categories  should  be  kept  within  limits:  More  than  just  a 
few  color  codes  and  the  information  processing  capacity  of  the 
user  is  exceeded.  Finally,  although  colors  do  not  have  standard 
meanings  like  words,  color  use  should  be  consistent  with  any 
existing  color-meaning  associations.  For  example,  in  the 
military,  green  or  blue  are  associated  with  friendly;  yellow  or 
white  with  unknown;  and  red  with  hostile. 

A  third  factor  to  be  considered  in  using  color  is  the  nature 
and  extent  of  its  effects  on  performance.  Although  color  has 
been  shown  to  enhance  performance  of  some  simple  tasks  such  as 
checking  'go/no  go'  on  system  status,  it  does  not  necessarily 
enhance  performance  of  more  complex  tasks  such  as  landing  an 
aircraft.  Also,  even  if  color  does  not  directly  affect 
performance,  there  is  some  evidence  that  it  may  reduce  subjective 
estimates  of  workload,  that  is,  how  hard  a  task  seems. 

2j^2 _ Q§yeioQment_of  _the_ExBerimentai_Coior_Disglay .  With 

the  above  factors  taken  into  consideration,  an  overall  color 
scheme  was  developed  for  the  Patriot  display  at  the  battery 
Engagement  Control  Station  (ECS).  (The  battalion  Information 
Coordination  Central  (ICC)  display  would  be  basically  the  same, 
but  was  not  specifically  developed  for  this  experiment.)  Before 
discussing  the  color  display  in  more  detail,  it  will  be  useful  to 
describe  the  basic  Patriot  display.  An  illustration  of  the 
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display  is  shown  in  Figure  2.1  along  with  the  swi tch- indicator 
panel ,  keyboard,  special  function  keys,  and  Joystick.  The 
situation  display  comprises  the  largest  area  on  the  display  and 
is  where  the  symbol,  shape,  and  line  coding,  the  brightness 
coding,  and  the  blink-rate  coding  are  employed.  It  is  a  stylized 
map  showing  moving  aircraft,  or  tracks,  against  a  static 
background  of  objects  such  as  search  and  track  sectors,  assets, 
safe  passage  corridors,  weapons  control  volumes,  and  restricted 
and  prohibited  volumes.  For  a  complete  listing  of  the  situation 
display  objects  and  their  coding,  refer  to  Table  2.1.  The  alert 
message  line  and  tabular  display  area  are  below  the  situation 
display  and  are  where  the  formatting  technique  is  employed.  The 
alert  message  line  (see  Figure  2.2  for  details)  comprises  a 
single  line  of  text  where  priority  messages  are  displayed  in 
succession.  The  tabular  display  area  comprises  a  rectangular 
area  where  tables  of  information  are  displayed  one  at  a  time. 

One  such  table  is  the  Engaged  Data  Table  (see  Figure  2.3)  which 
contains  the  To-Be-Engaged  Queue  (TBEQ)  and  the  Engaged  Queue. 

The  monochrome  version  of  the  Patriot  display  as  it  was 
represented  in  this  experiment  is  shown  in  Photograph  2.1. 

The  color  version  of  the  display  is  shown  in  Photograph  2.2. 
Color  was  essentially  added  to  the  monochrome  display  without 
changing  the  existing  coding  with  two  exceptions.  One  exception 
involved  the  use  of  the  color-fill  capability.  Using  that,  areas 
and  volumes  were  drawn  as  filled  polygons,  therefore  eliminating 
the  solid  or  dashed  line  coding.  The  other  exception  involved 
the  elimination  of  blink  coding.  Due  to  system  limitations  on 
multiple  blink  rates,  it  was  replaced  with  two  types  of  track 
symbol  modifiers  that  blinked  at  the  same  rate.  Instead  of  a 
track  blinking  at  6  Hz  when  it  was  hooked,  another  smaller 
version  of  the  track  symbol  (circle,  "U",  or  diamond)  blinked 
inside  the  track;  and  instead  of  a  track  blinking  at  3  Hz  when  it 
was  referenced  by  the  alert  message  line,  an  "X"  blinked  inside 
the  track. 

A  total  of  14  different  colors  was  used  in  the  situation 
display  including  black  for  the  background.  The  color  scheme  is 
consistent  with  military  color-meaning  associations:  Objects 
associated  with  hostile  information  were  represented  in  reddish 
or  pinkish  shades  and  those  associated  with  friendly  information 
in  shades  of  blue.  Other  objects  were  represented  in  relatively 
neutral  colors.  In  order  to  manage  the  potential  color  contrast 
problems,  the  background  objects  were  of  relatively  low 
intensity,  that  is,  not  very  bright,  and  fairly  unsaturated. 
Foreground  objects  were  of  high  intensity  and  highly  saturated. 

In  order  to  keep  the  number  of  color-coded  categories  within 
limits,  objects  with  the  same  shape  or  line  coding  and  objects 
grouped  logically  according  to  the  similarity  of  information  they 
provide  were  displayed  in  the  same  color.  The  number  and  letter 
designators  for  each  object  were  also  displayed  in  the  same  color 
as  the  object.  For  a  complete  listing  of  the  situation  display 
objects  and  their  respective  color  coding,  see  also  Table  2.1. 
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Ei9y!:§_2ii-  An  iHustration  of  the  Patriot  display.  TM-9-1430- 
§Q0ri0ri.  1983.  ■  ■ 
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Table  2.1.  Situation 

display  objects  and 

the  i  r 

respective  shape. 

or  line  coding,  blink 

coding,  and  experimental 

color  coding. 

_ Current  Coding 

Color  Coding 

Objects 

Shape  or  Line 

Blink 

Search  Sector 

\  / 

no 

1  i  ght  aqua<* 

Track  Sector 

\  \  /  ^ 

X 

no 

dark  aqua** 

Adjacent  Sector  Bounds 

no 

not  displayed 

Weapons  Control  Volumes  - 

Hold 

no 

1  i  ght  orange** 

Tight 

0 

no 

light  orange** 

Free 

no 

light  orange** 

Restricted  Volume 

no 

1  i  ght  pink** 

Prohibited  Volume 

^ 

no 

1  i  ght  pink** 

Host i 1 e  Origin 

« i 
»  / 

no 

not  displayed 

Friendly  Origin 

(  } 

no 

not  displayed 

Safe  Passage  Corridor 

no 

1  i  ght  blue** 

Defended  Area/Point 

Cwi 

no 

medium  blue** 

FEBA 

□ 

0 

Q 

no 

whi  te 

Masked  Terra i n 

FESA 

— <k  A 

X  /  s 

V  X 

no 

bright  orange 

Range  Rings 

'> — y 

no 

dark  purple 

Fire  Platoon  Flag 

'' 

no 

yel low-green 

Batta lion  Flag 

r 

no 

yel low-green 

Communications  Relay 

no 

yel 1 ow-green 

Group 


Nq1§*  The  shape  or  line  coding  is  not  to  scale.  'Yes'  under  the 
Blink  Coding  column  indicates  that  the  object  blinks  at  6  H2  when 
the  track  is  hooked  and  at  3  Hz  when  the  track  is  referenced  by 
the  Alert  Message  Line.  Color  codes  marked  with  an  asterisk  mean 
that  the  objects  were  solidly  colored,  that  is,  color-fill  was 
used.  Currently  and  In  the  experimental  color  display,  the 
background  is  black. 
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lab  1 e  2.1  con ' t . 
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Ei9yE§-2^i<  Alert  Message  Line  displaying  the  message  that  track 
number  002  is  a  priority  engagement  and  the  word  "more" 
indicating  that  additional  messages  are  waiting  to  be  displayed 
when  this  one  is  cleared. 


002  PRI  ENG 


MORE 


Eigyc§_2i3.  The  Engaged  Data  Table  with  entries  on  the  TBEQ 
(left  six  columns).  No  entries  are  shown  on  the  Engaged  Queue 
(right  four  columns).  TGTNO  =  target  number;  THRT  =  ATC  value; 
RT  =  release  time;  TLL  =  time  to  last  launch;  ENGSTAT  =  engaged 
status;  ID  =  identification;  SZ  =  size;  MA  =  missile  away;  and 
TGO  “  time  to  go  until  intercept. 


TGTNO 

THRT 

RT 

TLL  ENGSTAT 

lO/SZ 

-TGTNO  MA  TGO  SZ 

001 

1 

10 

30 

H 

002 

1 

0 

20 

H/2 

003 

9 

15 

99 

H 

The  alert  message  line  was  not  specifically  color-coded. 

For  this  area,  the  background  was  a  light  gray  and  the  text  was 
black.  This  color  combination  allowed  for  sufficient  contrast 
and  yet  was  relatively  easy  on  the  eyes.  Another  reason  for 
choosing  these  colors  was  so  that  the  contrast  was  in  the  same 
direction  as  the  situation  display:  dark,  saturated  foreground 
text  on  a  light  background.  Just  two  different  colors  were  used 
in  this  area,  black  and  light  gray. 

Many  different  tables  of  information  can  be  displayed  in  the 
tabular  display  area  but  color  coding  was  implemented  on  only  one 
table,  the  Engaged  Data  Table.  Otherwise,  for  all  tables,  the 
background  and  foreground  text  were  the  same  colors  as  for  the 
alert  message  line,  light  gray  and  black,  respectively.  On  the 
Engaged  Data  Table,  TBEQ  entries  were  color  coded  to  highlight 
critical  information.  When  Release  Time  (RT)  was  equal  to  zero, 
the  entry  appeared  in  red.  When  RT  was  greater  than  zero,  but 
Asset  Threat  Category  (ATC)  was  equal  to  one,  the  entry  appeared 
in  yellow.  (In  the  scenarios  used,  however,  this  situation  never 

occurred.)  All  other  entries  appeared  in  the  standard  text 
color,  black.  Thus,  in  this  area,  a  total  of  four  different 
colors  was  used;  black,  light  gray,  red,  and  yellow. 

On  the  following  page,  PhotograBh_2il .  Monochrome  version  of  the 
Patriot  display. 
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On  the  preceding  page,  PhotograQh_2^2.  Color  version  of  the 
Patriot  display.  Due  to  the  difficulty  in  photographing  raster 
scan  displays,  the  colors  in  the  photograph  are  degraded 
somewhat.  They  were  actually  substantially  brighter. 


3^0 _ EvaXuation_Methgd_and_Prgcedure 

The  evaluation  method  and  procedure  will  be  discussed  in 
five  sections:  3.1  Test  Operators,  3.2  Equipment,  3.3  Scenarios, 
3.4  Performance  Measures,  and  3.5  Testing.  Serving  as  an 
introduction,  an  overview  of  the  method  and  procedure  will  be 
given  first. 

The  performance  of  operators,  both  officers  and  enlisted 
personnel,  in  the  Weapons  Controller  position  at  the  ECS  was 
investigated.  The  Weapons  Controller  is  responsible  for  carrying 
out  engagement  decisions,  that  is,  decisions  to  fire  on  a  hostile 
track.  The  PTOS  was  operated  In  independent  and  autonomous  modes 
so  that  operators  were  on  their  own  in  deciding  what  to  engage 
and  when.  The  test  operators  were  given  twelve  different 
scenarios  with  three  levels  of  track  load  and  two  levels  of 
background  load  (e.g.,  the  number  of  volumes)  under  the  two 
display  conditions,  monochrome  and  color.  They  were  instructed 
to  defend  the  assets  to  the  best  of  their  abilities,  to  engage 
tracks  in  a  timely  fasHion,  and  to  conserve  missile  resources. 
Their  performance  was  assessed  through  mission  level  Summary 
Performance  Measures  (SPMs)}  more  specific  Task  Performance 
Measures  (TPMs);  and  the  Subjective  Workload  Assessment  Technique 
(SWAT).  Ratings  of  eye  comfort,  viewing  comfort,  image 
appearance,  and  overall  ease  of  viewing,  display  preference,  and 
general  comments  were  also  solicited. 

_ I§§t_QB§CilQC§ ♦  Trained  Patriot  operators  from  the  1st 

Patriot  Battal ion/43rd  Brigade  and  from  the  Identify  Friend,  Foe, 
or  Neut ra 1 / Jo int  Task  Force  at  Fort  Bliss,  Texas  part i c i pa ted  i n 
the  study.  The  criteria  for  being  trained  was  that  they  had 
completed  both  the  individual  school  training  and  the  collective 
field  training.  Twenty-four  operators,  12  officers  (14Es)  and  12 
enlisted  personnel  (24Ts)  were  tested.  Although  the  24T  is 
usually  designated  the  Weapons  Controller,  the  14E  is  cross- 
trained  as  the  Weapons  Controller  and,  therefore,  is  suitably 
familiar  with  the  task.  An  additional  seven  operators  were 
tested  as  a  part  of  preliminary  pilot  testing. 

_ Egyigment.  A  Raster  Technologies  Model  One/80  was  the 

display  equipment  used  for  the  study.  Its  key  capabilities  are 
real  time  interactive  graphics,  1280  x  1024  pixel  (picture 
element)  display  resolution,  multiple  display  windows,  polygon 
fill,  and  a  palette  of  16.7  million  colors.  Although  16.7 
million  different  colors  can  be  generated,  only  a  subset  of  that 
number  could  be  displayed  simultaneously  given  the  other  display 
requirements.  The  experimental  display  configuration  was  capable 
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of  displaying  a  maximum  of  27  different  colors  and  of  that,  a 
total  of  only  16  different  colors  was  actually  used.  So  instead 
of  a  simultaneous  display  capability,  the  16.7  million  colors 
provided  the  needed  flexibility  to  select  precise  shades. 

The  color  display  system  was  driven  by  the  pre-deployment 
build  PTOS  scenario  software  residing  in  the  host  computer,  the 
Gould  S.E.L.  32.77  under  operating  systems  1 .SB  and  2.2.  To 
complete  the  system,  a  keyboard  and  Joystick  were  used  as  input 
devices  and  specially  constructed  button  boxes  were  used  as  the 
switch-indicator  panels. 

3^3 _ Scenarios.  Scenario  development  was  accomplished  using 

a  menu-driven  scenario  development  procedure,  the  PTOS  Scenario 
Preprocessor.  Scenarios  are  created  by  specifying  the  background 
information:  the  location  of  the  assets,  fire  platoon, 

battalion,  forward  edge  of  the  battle  area  (FEBA) ,  weapons 
control  volumes,  hostile  identification  volumes,  safe  passage 
corridors,  and  origin  volumes;  and  the  track  or  foreground 
information:  speed,  altitude,  and  intended  target.  A  base 
scenario  was  built  based  on  realistic  tactical  information 
provided  by  Army  subject  matter  experts  in  an  existing  scenario. 
Acceptance  Test  Program  5  (ATP5).  The  training  and  test  scenario 
were  subsets  of  the  base  scenario.  Three  levels  of  hostile  track 
load  (see  Table  3.3.1)  and  two  levels  of  background  load  (see 
Table  3.3.2)  were  used.  Most  scenarios  had  some  hostile  tracks 
that  were  not  targeted  against  an  asset,  for  example,  tracks  that 
flew  into  the  area  of  coverage  but  then  turned  and  left  the  area. 
Track  entries  were  spaced  such  that  activity  was  spread 
throughout  the  six  minute  duration  of  the  scenario. 

3ifi _ P§Ef ormance_Measures .  Multiple  aspects  of  performance 

were  assessed  in  this  experiment  because,  as  pointed  out  in  the 
survey  of  color  research  (Section  2. 1 ) ,  the  effects  of  color  may 
not  be  seen  at  all  levels  of  performance.  In  the  following 
sections,  each  of  the  different  measures  will  be  discussed. 


Track  loads  and  characteristics. 


Track  Load 
Low 

Moderate 

High 


_ Hostiigs _  FrigQdiigs 

Total  Targeted  Not  Targeted 

Against  Assets  Against  Assets 


3  2  -  3  0  -1 

6  4-5  1-2 

12  7-10  2-6 


2-4 

2-4 

4-6 


I§bie_3i3^S.  Background  load  levels. 


Background  Display  Objects 
Assets 

Safe  Passage  Corridor 
Weapons  Control  Volume 
Hold 

Weapons  Control  Volume 
Tight 

Weapons  Control  Volume 
Free 

Prohibited  Volume 
Restricted  Volume 
Masked  Terrain 


Background  Load 
Simple  Complex 


6 

1 


6 

3 


1 


3 


1 


2 


1 

1 

1 

0 


2 

2 

2 

2 


Summary_Perforn3aDce_Measures_iSPMsi.  The  SPMs,  based  on 
earlier  ARI  work  (Hawley,  Howard,  S  Martellaro,  1982),  are 
aggregate  measures  designed  to  capture  overall  system 
performance — man  and  machine--at  the  mission  level.  Mission 
level  performance  is  measured  in  relation  to  bottom-line  or  long¬ 
term  goals.  The  SPMs  are  Probability  of  Kill  (PK) ,  Resource 
Conservation  (RC) ,  and  Defense  of  Assets  (DOA) .  1 

PK  is  defined  as  the  ratio  of  the  number  of  tracks  killed  to 
the  number  launched  on  for  a  given  scenario.  The  formula  is: 

PK  =  tk  /  tlo 

where  tk  =  tracks  killed  and 

tlo  =  tracks  launched  on. 

Note  that  PK  is  different  from  attrition,  which  is  the  ratio  of 
tracks  killed  to  those  available. 

RC  is  the  ratio  of  missiles  wasted  to  those  launched, 
corrected  for  system  failures  for  a  given  scenario.  It  is  scaled 
from  0,  all  missiles  wasted,  to  100,  no  missiles  wasted.  The 
formula  is: 


RC  =  Cl-(nmw  /  nml  -  nmf)3  100 
where  nmw  =  number  of  missiles  wasted, 

nml  =  number  of  missiles  launched,  and 
nmf  =  number  of  missiles  failed. 


1  The  SPM  Air  Space  Defense  as  developed  by  Hawley  et  al. 
(1982)  was  not  included  because  it  proved  to  be  insensitive  over 
the  range  of  values  seen  here.  Instead,  PK  was  calculated  as  an 
indication  of  how  well  the  air  space  was  defended. 
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DOA  is  the  ratio  of  assets,  weighted  by  priority,  that  were 
successfully  defended  to  those  that  were  scripted  for 
penetration.  The  number  actually  scripted  for  penetration  can  be 
smaller  than  the  number  of  assets  the  system  determined  were 
being  threatened  throughout  a  scenario.  DOA  is  scaled  so  that  a 
score  of  0  means  all  assets  were  penetrated  and  100  means  no 
assets  were  penetrated.  The  formula  for  calculating  DOA  is: 

C{  10  -  ap)  ♦♦  p  > 

-  H  100 

€(10  -  ap)  **  s  > 

the  number  of  assets  in  the  scenario; 
the  number  of  tracks  in  the  scenario; 
the  priority  value  of  the  asset; 

1  if  the  asset  is  penetrated,  and 
0  if  the  asset  is  not  penetrated;, 

1  if  the  asset  is  scripted  for 
penetration,  and 

0  if  the  asset  is  not  scripted  for 
penetrat i on. 

For  the  purposes  of  the  DOA  score,  the  asset  priority  values 
which  range  from  9,  the  lowest,  to  1,  the  highest,  are  subtracted 
from  10  so  that  a  high  priority  asset  will  have  a  high  weight  and 
a  low  priority  asset  will  have  a  low  weight. 

The  information  required  to  calculate  the  SPMs  was  obtained 
from  the  Performance  Assessment  Capability  (PAC) ,  a  data 
reduction  capability  implemented  on  the  PTOS.  Information  was 
collected  at  the  conclusion  of  each  scenario:  the  number  of 
engagements;  the  number  of  tracks  launched  on  and  killed;  the 
number  of  missiles  launched,  wasted,  and  failed;  and  the  number 
of  assets  penetrated.  These  data  were  transferred  by  tape  to  the 
API  VAX  11/730  computer  for  further  reduction  and  analysis. 

I§s!s_PgCfQC!!)§Dee_Measures_iTPMsi.  The  TPMs  measure 
performance  in  relation  to  immediate,  relatively  short-term 
goals.  They  provide  information  about  two  aspects  of  operator 
performance;  the  timeliness  of  engagements,  and  the  different 
activities  performed  throughout  the  scenario. 

Timeliness  of  engagements  was  calculated  in  two  ways;  as  the 
number  of  seconds  a  given  track  was  engaged  before  or  after  RT  = 
0,  and  8s  the  displayed  value  of  TLL  at  ^engagement.  These  two 
measures  can  be  thought  of  as  the  two  ends  of  the  engagement 
window.  RT  is  the  first  opportunity  for  engagement  of  a  given 
track,  and  TLL  is  the  last  opportunity  before  it  penetrates  an 
asset  or  leaves  the  area  of  coverage.  The  data  for  the 
calculations  were  drawn  from  the  PAC  data  collected  on  each  track 
at  one-second  time  hacks;  identification  (unknown,  friend  or 
hostile),  object  modifier  (no  status,  to-be-engaged,  engaged, 
under  k i 1 1 -assessment ) ,  RT ,  and  TLL.  The  data  were  reduced  and 
analyzed  on  the  VAX  11/730. 


DOA  =  1  - 


where  n  = 
m  = 
ap  = 
P  = 

s  = 
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In  calculating  engagement  time  in  relation  to  RT,  if  an 
engagement  occurred  prior  to  RT  =  0,  the  time  was  considered 
negative;  and  if  an  engagement  occurred  after  RT  =  0,  the  time 
was  considered  positive.  The  time  for  each  engagement  was 
included  in  the  calculation  even  if  it  was  a  repeat  engagement  or 
an  engagement  that  later  failed.  The  percent  of  the  total 
engagements  that  were  early  was  also  figured. 

The  calculation  of  mean  displayed  TLL  value  was  straight¬ 
forward  although  it  should  be  noted  that  it  does  not  have  a  one- 
to-one  correspondence  with  clock  time.  TLL  counts  down  from  99 
seconds  to  0  seconds,  but  may  Jump  from  99  to  some  intermediate 
number  and  then  count  down  sequentially  from  there.  Also,  TLL 
may  count  down  more  than  once  during  a  scenario  as  a  track  is 
paired  against  successive  assets. 

Information  about  the  different  activities  performed  was 
obtained  by  first  generating  a  list  of  possible  operator 
activities  from  IM_9rifi30-600-i0-i  (1983)  and  from  Outline 
IcaiDiD9_Pi§n_f Qr_Patrlot_ECS_Consgie_goerators  (Brett  &  Chapman, 
1983).  The  activities  were  then  categorized  based  on  their 
frequency  of  occurrence  during  the  performance  of  the  Weapons 
Controller  task.  The  resulting  17  categories  of  activities  are 
listed  in  Table  3.4.1.  Data  on  the  activities  were  collected 
using  an  Apple  lie  micro-computer.  The  experimenter  observed  the 
operator  during  the  test  and  "shadowed"  the  operator's  actions. 

In  other  words,  when  the  operator  began  an  activity,  the 
experimenter  pressed  a  corresponding,  coded  key  on  the  Apple, 
each  of  the  17  activities  having  been  assigned  to  a  different 
key.  For  each  key-press,  the  code  for  the  key  and  the  time  were 
recorded  at  one-second  time  hacks.  Thus,  the  time  spent  on  each 


I§bie_3i4ii.  List  of  activities  possible  to  be  performed  by 
Patriot  Weapons  Controllers  at  the  ECS. 


1.  Select  and  inspect  Track  Amplification  Data  Table 

2.  Select  and  inspect  Engagement  Data  Table 

3.  Select  and  inspect  other  tables 

4.  Select/deselect  displayed  track  data 

5.  Drop  track  from  display 

6.  Change  display  size  or  offset  display 

7.  Engage 

8.  Request  hold  fire,  cease  fire,  or  engage  hold 

9.  Select  method  of  fire 

10.  Perform  IFF  (Identify  Friend  or  Foe)  activity 

11.  Perform  other  identification  activity 

12.  Acknowledge  and  inspect  alert 

13.  Cursor  hook  a  track 

14.  Number  (num)  hook  a  track 

15.  Sequence  hook  track  from  the  TBEQ 

16.  Cancel  a  hook  on  a  track 

17.  Otherwise  monitor  the  display 
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instance  of  a  given  activity  is  calculated  by  subtracting  the 
start  time  of  the  activity  from  the  start  time  of  the  next 
activity.  The  number  of  instances  and  the  mean  time  per  instance 
of  each  of  the  activities  were  calculated  for  each  scenario.  The 
raw  data  were  recorded  on  floppy  diskettes  and  transferred  to  the 
VAX  11/730  where  they  were  reduced  and  analyzed. 

iyfeJ§ctive_Wgrkioad_AssessmeDt .  The  Subjective  Workload 
Assessment  Technique  (SWAT)  (Reid,  Shi ngl edecker  S  Eggemeier, 
1981)  used  in  this  experiment  was  developed  by  the  USAF  Aerospace 
Medical  Research  Laboratory  at  Wr i ght-Patterson  Air  Force  Base. 

It  provides  an  overall  subjective  assessment  of  workload  based  on 
assessments  across  three  dimensions;  time  load,  mental  effort 
load,  and  psychological  stress  load,  each  of  which  can  vary 
according  to  three  levels  of  load.  In  addition  to  considering 
the  mul t i dimensional i ty  of  workload,  this  technique  takes  into 
consideration  individual  differences  in  subjective  assessment. 

The  procedure  for  obtaining  the  subjective  workload  ratings 
has  three  main  steps.  First,  each  operator  rank  orders  the  27 
combinations  of  the  three  levels  of  the  three  dimensions  by 
sorting  a  deck  of  27  cards.  (See  Appendix  A  for  an  example  of 
one  of  the  27  cards.)  The  rank  orders  are  then  subjected  to 
computer  analysis  (c.f.,  Nygren,  1982)  to  arrive  at  a  single 
scale,  0,  low  workload,  to  100,  high  workload,  for  the  group. 

(The  scale  arrived  at  here  is  included  in  Appendix  A.)  The 
second  step  is  to  obtain  workload  ratings,  1,  2,  or  3,  for  each 
of  the  three  dimensions,  on  the  task  of  interest.  In  this 
experiment,  ratings  were  obtained  from  each  operator  after  each 
scenario.  (The  SWAT  rating  scales  used  in  this  experiment  are 
shown  in  Appendix  A.)  The  third  step  is  to  reduce  those  ratings 
to  a  single  number  on  the  basis  of  the  derived  scale.  These 
scores  are  then  subjected  to  further  analysis. 

SubJective_OBinions .  Subjective  ratings  or  opinions  were 
solicited  from  the  operators  on  several  factors  important  for 
comparing  the  monochrome  and  color  displays.  Ratings  were 
obtained  on  two  dimensions  of  eye  comfort,  two  dimensions  of 
viewing  comfort,  five  dimensions  of  image  appearance,  and  overall 
display  ease  of  viewing,  based  on  dimensions  identified  in  work 
published  by  Christensen,  Baggen,  and  Snyder  (1985).  (The 
ratings  were  made  on  the  rating  scales  shown  in  Appendix  B. )  A 
display  preference  and  general  comments  about  the  colors  and 
suggestions  for  changes  were  also  obtained. 

3^5 _ listing.  Each  test  operator  attended  two  testing 

sessions,  a  small  group  session  with  up  to  four  test  operators 
and  an  individual  session.  The  first  stage  of  SWAT,  the 
introduction  and  card  sorting,  and  display  and  equipment 
familiarization  were  accomplished  in  the  small  group  sessions. 

It  was  important  to  ensure  display  and  equipment  familiarization 
since  there  are  some  configural  differences  between  the  color 
display  system  and  the  tactical  Patriot  system. 
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Testing  with  the  display  was  conducted  in  the  individual 
sessions.  First,  each  operator  worked  through  two  practice 
scenarios,  each  lasting  ten  minutes  on  the  scenario  clock,  one  in 
monochrome  and  one  in  color.  Half  the  operators  practiced  with 
the  displays  ordered  monochrome-color;  half  with  the  displays 
ordered  color-monochrome.  During  the  practice,  the  operator  was 
permitted  to  ask  questions  of  the  experimenter. 

Next,  each  operator  worked  through  the  12  test  scenarios, 
each  lasting  six  minutes  on  the  scenario  clock.  All  operators 
saw  the  12  scenarios  in  the  same  order,  which  was  random  with  the 
control  that  the  two  test  halves  were  balanced  with  respect  to 
track  load  and  background  load.  All  the  operators  worked  through 
six  scenarios  in  monochrome  and  six  in  color.  Half  the  operators 
were  tested  with  the  displays  ordered  monochrome-color;  half  with 
the  displays  ordered  color-monochrome. 

Assignment  of  operators  to  the  training  and  testing  orders  • 
was  control  1 ed  such  that  equal  numbers  of  officers  and  of 
enlisted  personnel  were  in  each  experimental  condition.  Table 
3.5.1  shows  the  resulting  assignments. 

The  SUIAT  ratings  were  made  after  each  scenario.  The  ratings 
of  eye  comfort,  viewing  comfort,  image  appearance,  and  overall 
ease  of  viewing  were  made  after  the  first  six  scenarios  (i.e., 
after  all  of  one  display  type),  and  again  after  the  second  six 
(i.e.,  after  all  of  the  other  display  type).  The  choice  of 
display  preference  and  the  comments  were  made  at  the  end  of  the 
entire  testing  session. 


I§ble_3^5^1.  Number  of  test  operators,  officers  and  enlisted  in 
each  of  the  testing  order  and  training  order  conditions. 


Training  Order 
Mono-Col  or 


Color-Mono 


Testing  Order 


Mono-Color 

3  14E 
3  24T 

3  14E 
3  24T 


Co  I or-Mono 

3  14E 
3  24T 

3  14E 
3  24T 


4j.O _ Result  s_aDd_DiscussigD_gf_the_Evaluatign 

Data  analysis  for  all  of  the  measures  proceeded  in  two  main 
steps.  First,  the  measures  were  calculated  from  the  raw  data. 
Then,  the  measures  were  subjected  to  a  repeated  measures  analysis 
of  variance.  Because  of  the  large  number  of  experimental 
variables  in  the  analysis  model  (see  Table  4.1  for  a  complete 
listing),  a  discussion  of  all  of  the  individual  effects  and 
interactions  is  beyond  the  scope  of  this  report.  Therefore,  the 
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I§feie_4j,]..  Experimental  variables. 


Personnel ; 
Training  Order: 
Testing  Order: 
Display: 

Track  Load: 
Background  Load; 


Off i cer 

Mono-Color 

Mono-Color 

Monochrome 

Low 

Simple 


Enl i sted 
Co  1 or-Mono 
Color-Mono 
Color 

Moderate  High 

Complex 


emphasis  throughout  this  section  will  be  on  the  overall  pattern 
of  results  as  they  pertain  to  the  evaluation  of  the  usefulness  of 
color  in  the  Patriot  display.  A  more  complete  discussion  of  the 
results  will  appear  in  an  ARI  technical  report. 

The  predominant  effect  across  the  measures,  the  effect  of 
the  order  in  which  the  displays  were  presented  to  the  operators, 
warrants  further  comment  before  proceeding.  Two  reasons  underlie 
the  effect.  One  is  the  differential  degrees  of  familiarity  with 
the  two  display  types:  The  operators’  pre-experiment  experience 
was  entirely  with  the  Patriot  monochrome  display.  Thus,  at  the 
beginning  of  the  testing  session,  even  though  all  the  operators 
had  Just  practiced  on  both  the  monochrome  display  and  the  color 
display,  the  monochrome  display  was  highly  familiar  and  the  color 
display  was  only  somewhat  familiar.  The  second  reason  is  that, 
since  some  aspects  of  the  simulation  itself  were  different  from 
the  actual  Patriot  (e.g.,  the  swi tch— indicator  response  time, 
etc.),  two  new  things  were  actually  being  presented  to  the 
operators  in  this  experiment — the  simulation  system  and  the  color 
display.  Operators  in  the  Mono-Color  test  order  were  able  to 
gain  familiarity  and  expertise  with  the  simulation  system  before 
being  tested  on  the  color  display,  whereas  the  operators  in  the 
Color-Mono  test  order  were  tested  on  the  color  display  at  the 
same  time  they  were  gaining  familiarity  and  expertise  with  the 
simulation  system.  So,  the  effects  of  testing  order  must  be 
considered  in  order  to  properly  understand  the  effects  of  the 
color  display. 

In  reading  this  section,  several  other  things  should  be  kept 
in  mind.  For  one,  most  of  the  effects  reported  here  are  small 
and  should  be  interpreted  as  such  for  application  to  a  practical 
situation.  Also,  although  the  operators  are  trained,  their 
console  experience  was  not  necessarily  recent  and  so  their  level 
of  performance  probably  was  lower  than  would  be  expected 
otherwise.  Finally,  with  this  particular  system  configuration, 
real  time  was  degraded  somewhat  with  increasing  track  loads. 

Thus,  the  track  load  manipulation  must  be  interpreted  cautiously. 

The  results  for  each  type  of  measure  will  be  reported  in  the 
separate  subsections  which  follow.  For  comparison,  the  system 
measures  will  be  reported  where  applicable.  They  were  computed 
from  data  obtained  when  running  the  system  in  automatic  mode  with 
no  operator  intervention. 
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_ SyiDfDQCii-PgCf QC!!!§DS§_y6§§yC§l •  2  To  preview  the  results 

found  with  the  SPMs:  Overall,  operator  performance  was  below 
that  of  system  performance.  Considering  the  operator  PK,  RC  and 
DOA  scores  together,  the  improvement  in  PK  over  time  was  at  the 
expense  of  a  lower  RC  score,  that  is,  more  aircraft  were  killed 
but  more  missiles  were  wasted.  Moreover,  increased  PK  did  not 
result  in  better  DOA.  Apparently  the  tracks  were  killed  after 
the  assets  were  penetrated.  Across  all  three  SPMs,  performance 
with  the  color  display  was  slightly  lower  than  that  with  the 
monochrome  display;  however,  this  difference  was  attenuated 
somewhat  by  the  second  test  half  for  PK  and  DOA,  presumably  due 
to  the  operators  simply  gaining  experience  with  the  color  display 
and  overcoming  the  poor  performance  with  the  color  display  in  the 
first  test  half.  This  "catching  up"  done  by  the  operators  who 
had  the  Color-Mono  test  order  was  evidenced  as  improvement  in  PK 
and  no  further  decrement  in  RC.  DOA  showed  a  different  pattern: 
Performance  was  initially  higher  with  the  color  display  but 
decreased  more  over  time. 

CCQbabXJ_ity_of _KXii .  The  mean  number  of  tracks  engaged  for 
each  scenario  is  shown  in  Table  4.1.1  along  with  the  minimum  and 
maximum  number  of  tracks  engaged.  The  number  of  tracks  engaged 
varied  across  scenarios  because  of  the  three  levels  of  the  track 
load  variable.  Note,  too,  that  the  number  of  tracks  engaged 
varied  within  a  given  scenario.  This  occurred  because  sometimes 
low  threat  tracks  were  engaged  unnecessarily  and  other  times  high 
threat  tracks  were  not  engaged  when  they  should  have  been. 

The  mean  operator  PK  was  0.82  and  the  system  PK  was  0.98. 
System  PK  was  less  than  1  because  in  some  scenarios  there  were 
so-called  distracter  tracks,  the  tracks  that  entered  the  zone  of 
coverage,  were  engaged,  but  then  turned  and  left  the  zone  before 
they  could  be  successfully  killed. 

Overall,  PK  was  greater  with  the  monochrome  display,  0.83, 
than  with  the  color  display,  0.81,  a  small  but  statistically 
reliable  difference.  Interactions  of  other  variables  underlie 
this  main  effect:  With  the  Mono-Color  test  order,  PK  improved 
only  slightly  over  time,  that  is,  from  the  first  display  type  to 
the  second.  But  with  the  Color— Mono  test  order,  PK  improved  more 
substantially  (see  Table  4.1.2).  With  practice,  then,  PK 
improved,  but  the  improvement  was  greater  for  those  operators 
going  from  the  color  to  the  monochrome  display  than  for  those 
going  from  the  monochrome  to  the  color  display.  This  is  the  test 
order  effect:  Operators  who  were  tested  first  on  the  color 
display  had  to  deal  with  both  the  simulation  system  and  the  color 
display.  Their  performance  was  worse  in  the  beginning,  but 
improved  with  practice  to  about  the  same  level  as  the  performance 
of  operators  tested  in  the  Mono-Color  order. 


2  Data  on  the  SPM's  were  lost  for  one  operator  for  one 
scenario  due  to  an  error  in  the  execution  of  the  PAC. 
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I§bie_4iiii.  Mean  number  of  tracks  engaged  for  each  scenario 
along  with  the  minimum  and  maximum  number  of  tracks  engaged, 
according  to  track  load. 


Scenario 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1  1 

12 

Track 

Load 

M 

L 

H 

M 

L 

H 

L 

H 

M 

H 

M 

L 

Mean 

6.8 

3.9 

13.3 

6.9 

3.0 

13.5 

3.0 

13.0 

6.0 

12.6 

6.8 

3.9 

Min 

6 

3 

1  1 

6 

3 

1  1 

3 

12 

5 

1 1 

4 

3 

Max 

7 

4 

14 

7 

3 

14 

3 

14 

7 

13 

7 

4 

Note. 

L  = 

1  ow. 

M  = 

moderate. 

H  = 

high. 

I§bie_4iii2.  Mean  PK  for  the  monochrome  and  color  displays  for 
the  two  test  orders,  Mono-Color  and  Color-Mono. 

Test  Half 
First  Second 

Test  Order  Mono-Color  0.81  0.85 

Color-Mono  0.78  0.86 


Resource_Conservation.  Mean  operator  RC  for  the  12 
scenarios  was  77.73;  system  RC  is,  by  definition,  100.  RC  was 
slightly  greater  for  the  monochrome  display  than  for  the  color 
display  (see  Table  4.1.3).  Again,  as  with  PK,  this  is  a  small 
but  reliable  effect.  Note  that  fewer  missiles  were  launched  with 
the  color  display  than  with  the  monochrome  display  but  that  more 
missiles  were  wasted  which  resulted  in  a  lower  RC  score.  The 
pattern  of  RC  scores  over  time  can  be  interpreted  as  a  test  order 
effect  (see  Table  4.1.4)  although  the  effect  was  not 
statistically  significant.  Initially  performance  was  slightly 
lower  with  the  color  display,  but  over  time,  performance  was  at 
the  same  level  for  both  display  types. 


The  numbers  of  missiles  launched  and  wasted  and 
mean  R(^  score  according  to  display  type. 


M i ss i 1 es 

Launched  Uasted 

Display  Monochrome  8.51  1.81 

Color  8.25  1.97 


RC 

78.79 

76.79 
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Mean  RC  the  monochrome  and  color  displays  for  the 
two  test  orders,  Mono-Color  and  Color-Mono. 


Test  Half 
First  Second 

Test  Order  Mono-Color  80.01  77.19 

Color-Mono  76.26  77.56 


lifeig-SiiiS.  Mean  DOA  for  the  monochrome  and  color  displays  for 
the  two  test  orders,  Mono-Color  and  Color-Mono. 

Test  Half 
First  Second 

Test  Order  Mono-Color  86.71  79.08 

Color-Mono  88.15  83.28 


QefgDS§_Qf_Assets.  Mean  operator  DOA  was  84.50  and  system 
DOA  was  100.00.  Overall,  there  was  no  difference  in  DOA  between 
scenarios  seen  on  the  monochrome  display,  85.00,  and  those  seen 
on  the  color  display,  84.00.  When  the  effect  of  test  order  is 
examined  (see  Table  4.1.5),  DOA  was  actually  slightly  better  with 
the  color  display  than  with  the  monochrome  display  during  the 
first  test  half  and  worse  during  the  second  test  half. 

_ I§sk_Performance_Measures.  The  main  finding  with 

respect  to  the  TPMs  is  that,  overall,  operator  performance 
deviated  substantially  from  system  performance.  On  the  one  hand, 
operators  engaged  nearly  40%  of  the  tracks  early  whereas  the 
system  engaged  virtually  no  tracks  early,  early  defined  as  before 
RT  =  0.  On  the  other  hand,  operators  engaged  the  other  60%  of 
the  tracks  later  than  the  system  by  a  factor  of  40,  with  the 
result  that  less  time  was  remaining  in  which  to  engage  the  tracks 
and  the  tracks  were  more  threatening. 

Further,  there  were  no  major  performance  differences  as  a 
function  of  display  type.  The  color  display  neither  helped  nor 
hurt  task  performance  as  defined  here.  In  two  specific  cases 
though — the  percentage  of  early  engagements  and  taking  advantage 
of  automatic  hooking  by  acknowledging  certain  alerts — there  is 
some  evidence  that  the  color  display  made  performance  more 
ef f i c i ent . 

Ii!DgiiDgss_Qf_ingagements.  On  average,  the  operators 
engaged  tracks  27.37  seconds  after  RT  =  0  whereas  the  system 
engaged  tracks  only  1.44  seconds  after.  The  mean  displayed  value 
of  TLL  at  engagement  for  the  operators  was  15.85,  and  for  the 
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system  it  was  25.11.  In  other  words,  the  operators  engaged 
farther  into  the  engagement  window  on  average  than  did  the  system 
so  that  there  was  less  time  remaining  before  the  last  possible 
opportunity. 

In  order  to  more  completely  understand  the  time  to  engage  in 
relation  to  RT  =  0,  it  should  be  clear  that  both  engagements 
occurring  before  RT  =  0,  and,  therefore,  having  negative  times, 
and  those  occurring  after  RT  =  0  are  included  in  the  calculation 
of  the  overall  time  to  engage.  Breaking  out  those  times 
separately  (see  Table  4.2.1),  it  becomes  apparent  that  the  time 
to  engage  was  widely  distributed,  from  an  average  of  -20.64 
seconds  to  +57.63  seconds.  In  fact,  39.24%  of  the  engagements 
occurred  before  RT  =  0.  One  consequence  of  early  engagements  was 
repeat  engagements  later  in  the  engagement  window  when  TLL  was 
lower.  Such  was  the  case  here  where  operators  had  an  average  of 
1.79  repeat  engagements  per  scenario. 

The  figures  for  times  to  engage  and  percent  early 
engagements  as  a  function  of  display  type  are  shown  in  Table 
4.2.2.  The  pattern  reveals  that  mean  times  to  engage  before  RT  = 
0  and  after  RT  =  0  for  the  color  display  were  closer  to  0  than 
the  times  for  the  monochrome  display.  The  percentage  of  early 
engagements  was  also  smaller  with  the  color  display  than  with  the 
monochrome  display.  Although  none  of  these  results  were 
statistically  significant,  the  pattern  is  intriguing.  It  appears 
that,  with  the  color  display,  operators  were  more  likely  to  wait 
for  RT  =  0  because  it  was  highlighted  in  red.  Mean  TLL  was  not 
different  as  a  function  of  display  type,  15.77  for  the  monochrome 
display,  and  15.93  for  the  color  display. 


I§bie_4i2^i.  Mean  time  to  engage  in  relation  to  RT  =  0  and  the 
percent  of  the  early  engagements  for  the  operators  and  for  the 
system . 


_ Iif!)e_io_iD9§9e _ 

Before  RT=0  After  RT=0 
Operators  -20.64  57.63 

System  -  1.44 


Percent 

_Eariy 

39.24 

0.00 


I§bie_4i2i2.  Mean  time  to  engage  in  relation  to  RT  =  0  and  the 
percent  of  early  engagements  as  a  function  of  display  type. 


Percent 

_ Ii2!e_to_EDgage _  _i§riy 

Before  RT=0  After  RT=0 
-21.48  59.19 

-19.75  57.94 


Monochrome 

Color 


40.90 

37.57 


Activj_ties_Performed.  The  mean  frequency  of  instances  per 
scenario  for  each  of  the  17  activities  is  shown  in  Table  4.2.3. 
Due  to  the  low  frequency  of  many  of  the  activities,  only  the  five 
highest  frequency  activities  will  be  reported  further.  Those 
five  activities  are  alert  acknowledge,  engage  data  table,  engage, 
hook,  and  monitor.  Hook  is  the  sum  of  the  three  different  types 
of  hooking,  num  hook,  sequence  hook  and  cursor  hook. 

The  type  of  display  alone  made  no  significant  difference  for 
any  of  the  five  activities  as  measured  by  either  the  number  of 
instances  or  the  mean  time  per  instance.  The  figures  are  listed 
in  Table  4.2.4.  Table  4.2.5  shows  the  same  data  broken  out  by 
both  display  and  test  order.  For  each  of  the  activities,  there 
were  fewer  instances  during  the  second  test  half  than  during  the 
first.  For  the  alert  acknowledge,  engaged  data  table,  engage, 
and  hook  activities,  each  instance  of  the  activity  took  less  time 
during  the  second  half  than  during  the  first.  However,  for  the 
monitoring  activity,  each  instance  took  more  time  during  the 
second  half  than  during  the  first.  In  other  words,  there  were 
fewer  separate  instances  of  monitoring,  but  each  one  lasted 
longer. 

A  closer  examination  of  the  number  of  instances  of  alert 
acknowledge  and  hook  reveals  an  interesting  relation  between  the 
two  as  a  function  of  test  order.  Under  the  Mono-Color  test 
order,  there  was  a  decrease  of  only  0.81  in  the  number  of  alert 
acknowledge  instances,  but  a  decrease  of  2.03  in  the  number  of 


I§bXe_4^2i3.  Mean  number  of  instances  per  scenario  for  each  of 
the  17  activities. 


Activity  or  Mean  Number 

Switch-Indicator  Press  of  Instances 


Track  Amp  Table  0.77 
Engaged  Data  Table  13.77 
Other  Tables  0.25 
Track  Data  I .30 
Drop  Track  0.00 
Size  or  Offset  0.52 
Engage  10.03 
Hold  Fire  0.00 
Method  of  Fire  0.36 
IFF  0.08 
ID  0.00 
Alert  Acknowledge  12.36 
Cursor  Hook  0.60 
Numeric  Hook  2.64 
Sequence  Hook  7.60 
Cancel  Hook  0.55 
Monitor  26.94 
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hook  instances.  The  pattern  is  exactly  reversed  under  the  Color- 
Mono  test  order:  there  was  a  decrease  of  2.14  in  the  number  of 
alert  acknowledge  instances,  but  a  decrease  of  only  0.86  in  the 
number  of  hook  instances.  Both  of  these  interactions  are  highly 
significant.  One  possible  explanation  for  these  effects  is  that 
under  the  Mono-Color  test  order,  with  the  experience  gained  by 


Tabie_4i2^4.  Mean  number  of  instances  and  mean  time  per  instance 
per  scenario  as  a  function  of  display  type  for  the  five  primary 
act i vi t i es . 


Alert  Acknowledge 

Engaged  Data  Table 

Engage 

Hook 

Moni tor 

Note.  Mean  time  per 


Instances 


Display 


Monochrome 

Col  or 

1  1  .92 

12.59 

13.88 

13.65 

10.03 

10.02 

11.12 

10.53 

26.89 

26.98 

Ii!D§_D§C_iD§i§DSg 


Di splay 


Monochrome 

Color 

2.50 

2.52 

2.70 

2.82 

2.43 

2.53 

3.21 

3.29 

10.66 

10.97 

instance  is  shown  in  seconds. 


I§bie_4i2j,5 .  Mean  number  of  instances  and  mean  time  per  instance 
per  scenario  as  a  function  of  display  type  and  test  order  for  the 
five  primary  activities. 


Instances 

T i me  per 

Instance 

Alert  Acknowledge 

Test 
F  i  rst 

Half 

Second 

Test 
F  i  rst 

Half 

Second 

Test  Drder 

Mono-Color 

13.06 

12.25 

2.56 

2.44 

Engaged  Data 

Color-Mono 

Table 

12.93 

10.79 

2.59 

2.43 

Test  Drder 

Mono-Color 

14.50 

11.88 

2.85 

2.71 

Engage 

Color-Mono 

15.43 

13.26 

2.93 

2.55 

Test  Drder 

Mono-Color 

10.26 

9.61 

2.56 

2.49 

Hook 

Color-Mono 

10.43 

9.81 

2.57 

2.31 

Test  Drder 

Mono-Color 

11.00 

8.97 

3.42 

3.14 

Moni tor 

Color-Mono 

12.10 

11.24 

3.44 

3.00 

Test  Drder 

Mono-Color 

28.56 

24.61 

10.23 

12.65 

Color-Mono 

29.35 

25.22 

9.29 

11.08 

Note.  Mean  time  per  instance  is  shown  in  seconds. 
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the  time  the  color  display  was  used,  the  operators  took  advantage 
of  the  automatic  hooking  feature  of  acknowledging  certain  kinds 
of  alerts  thereby  keeping  the  instances  of  alert  acknowledge  high 
while  reducing  the  instances  of  hooking.  In  fact,  in  one 
scenario,  one  operator  usdd  only  the  automatic  hooking  feature. 
Under  the  Color-Mono  test  order,  however,  even  with  the 
experience  gained  over  time,  the  operators  did  not  take  as  much 
advantage  of  the  automatic  hooking  when  using  the  monochrome 
display.  Thus,  those  operators  kept  the  number  of  instances  of 
hooking  high  while  reducing  the  number  of  instances  of  alert 
acknowledge  and,  presumably,  missing  some  alerts. 

_ §y^J§£iive_Workload_Assessment .  The  higher  the  rating, 

the  more  difficult  the  scenario  was  determined  to  be.  The  track 
load  variable  is  a  check  that  subjective  workload  did  vary 
significantly:  workload  ratings  were  5.S8  for  low,  24.89  for 
moderate,  and  40.79  for  high  track  load  scenarios.  Although 
workload  increased  with  increasing  track  load,  even  the  high 
track  load  scenarios  were  rated  below  the  middle  of  the  scale. 

Workload  ratings  for  the  scenarios  presented  with  the 
monochrome  display,  24.70,  did  not  differ  significantly  from  the 
ratings  for  the  scenarios  presented  with  the  color  display, 

22.61.  There  was,  however,  an  interaction  of  test  order  and 
display.  These  figures  are  shown  in  Table  4.3.1.  It  can  be  seen 
that  only  for  the  Mono-Color  test  order  did  workload  ratings 
decrease  over  time.  Thus,  experience  on  the  system  and  the  color 
display  combined  to  make  the  task  seem  easier  to  the  operator. 

The  combination  of  experience  and  the  monochrome  display  did  not 
make  the  task  seem  easier. 

Further,  workload  ratings  differed  as  a  function  of  display, 
test  order  and  track  load.  Under  both  test  orders,  workload 


Table  4.3.1.  Mean  SWAT  worklnarl 

rati ngs 

for  the  two  test  orders 

for  the  two  disp 

lay  types. 

Test 

Half 

F  i  rst 

Second 

Test  Order 

Mono-Color 

Track  Load 

Low 

9.58 

4.28 

Moderate 

22.50 

20.53 

High 

43.68 

30.31 

A1  1  Track  Loads 

25.25 

18.38 

Color-Mono 

Track  Load 

Low 

5.15 

2.11 

Moderate 

26.45 

30.10 

High 

48.94 

40.23 

All  T rack  Loads 

26.85 

24.14 

Note.  The  scale 

is  from  0,  low 

to  100,  high. 

ratings  decreased  over  time  for  low  track  loads.  However,  under 
the  Color-Mono  test  order  workload  ratings  actually  increased  for 
moderate  track  loads  and  decreased  by  about  8  points  for  high 
track  loads;  and  under  the  Mono-Color  test  order,  workload 
ratings  stayed  about  the  same  for  moderate  track  loads  and 
decreased  by  more  than  13  points  for  high  track  loads.  Thus,  the 
reduction  of  workload  ratings  seen  as  a  result  of  the  combination 
of  experience  on  the  system  and  the  color  display  was  even 
greater  with  increasing  track  load.  This  supports  the  notion 
that  subjective  advantages  of  the  color  display  are  greater  the 
more  cluttered  the  display. 

Also  of  interest  is  the  result  that  the  officers  rated  the 
scenarios  as  more  difficult  than  the  enlisted  operators,  39.00 
compared  to  18.31.  This  is  probably  due  to  the  fact  that  the 
officers  are  only  secondarily  trained  in  the  weapons  controller 
task  whereas  the  enlisted  operators  are  primarily  trained  in  the 
task. 


_ SubJectiye_QBinions .  Previewing  the  results  of  the 

subjective  opinion  data  discussed  in  this  section:  Overall,  the 
ratings  of  eye  and  viewing  comfort,  image  appearance  and  overall 
ease  of  viewing  revealed  no  differences  between  the  monochrome 
and  the  color  displays  although  they  do  indicate  some  differences 
in  what  is  distinctive  or  salient  about  them.  Moreover,  although 
the  color  display  was  preferred  by  only  slightly  more  operators 
than  the  monochrome  display,  the  comments  of  the  operators 
preferring  the  monochrome  display  suggest  that  some  specific 
aspects  of  the  color  display  were  preferable  to  the  monochrome 
display. 

B§tiD9§_Qf  _Eye_Comf  ort  j._yXewing_Cgmfort  jt._Image_ABQearance 
§Dd_Qyeraii_Ease_gf_yiewing.  Mean  ratings  are~shown~in  Table 
4.4.1.  A  rating  of  1  is  low;  a  rating  of  5  is  high.  Ratings  for 
both  displays  were  all  higher  than  3,  that  is,  higher  than  the 
middle  of  the  scale.  Further,  the  ratings  for  the  monochrome 
display  and  those  for  the  color  display  were  very  close.  The 
statistical  analyses  revealed  no  reliable  differences  as  a 
function  of  display  type  on  any  of  the  scales. 

The  intercorrelations  of  the  ratings  did  reveal  some 
differences  in  how  the  operators  considered  the  monochrome  and 
color  displays.  The  intercorrelation  matrices  for  the  ratings  of 
the  monochrome  display  and  for  the  color  display  are  shown  in 
Appendix  C.  To  summarize  what  the  intercorrelations  of  the 
ratings  Indicate,  the  salient  aspects  of  the  monochrome  display 
tended  to  be  object  specific  (e.g.,  image  distinctiveness  and 
stability)  whereas  the  salient  aspects  of  the  color  display 
tended  to  be  more  global  (e.g.,  ease  of  reading  and  color 
contrast).  Further,  although  the  same  dimensions  of  eye  comfort 
and  image  appearance  were  related  to  overall  ease  of  viewing  for 
both  display  types,  the  dimensions  of  vision  comfort  were  also 
related  to  overall  ease  of  viewing  with  the  color  display. 
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I§bie_ffi4j.i.  Ratings  for  the  two  dimensions  of  eye  comfort,  two 
dimensions  of  vision  comfort,  five  dimensions  of  image  appear¬ 
ance,  and  overall  ease  of  viewing  for  the  two  display  types. 

Di splay 


Monochrome 

Co  1  or 

Eye  Comfort 

T i red/rested 

3.50 

3.79 

Burning/norma  1 

4.17 

4.25 

Vision  Comfort 

B 1 urry/c 1  ear 

4.42 

4.21 

Changing/cons i stent 

4.33 

4.21 

Image  Appearance 

FI i cker i ng/stabl e 

4.04 

3.96 

Low/high  contrast 

3.50 

3.58 

Fuzzy/clear 

4.33 

4.33 

Hard/easy  to  read 

4.17 

3.71 

Confused/di  St i net 

3.42 

3.92 

Overall  Ease  of  Viewing 

Hard/easy  to  view 

3. 88 

3.71 

The  two  words  or  phrases  shown  for  each  dimension  are  the 
endpoints  of  the  rating  scales. 


QisBi§i'_Pcgference_and_Cornments .  The  color  display  was 
preferred  by  only  a  slim  margin,  13  to  11,  over  the  monochrome 
display.  Given  the  relatively  small  number  of  operators 
participating  in  the  test,  this  cannot  be  called  a  decisive  vote. 
The  breakout  according  to  officer  and  enlisted  personnel  is  shown 
in  Table  4.4.2.  The  officers  slightly  favored  the  monochrome 
display  and  the  enlisted  personnel  favored  the  color  display. 

More  instructive  than  the  simple  display  preference  are  the 
operators'  comments  about  why  they  preferred  the  display  they 
did.  A  complete  listing  of  all  operator  comments  is  contained  in 
Appendix  D.  Only  an  overview  will  be  presented  here. 

The  operators  who  preferred  the  monochrome  display  said  that 
it  was  easier  to  pick  out  specific  targets  or  objects  on  the 
monochrome  display  than  on  the  color  display.  They  said  that  it 
was  easier  to  distinguish  the  shape  coding.  Four  of  those 
operators  mentioned  that  the  color/brightness  contrast  was 
better,  clearer  with  the  monochrome  display.  Further,  some  said 
that  the  monochrome  was  'easier  on  the  eyes’,  that  the  color 
display  had  "too  much  action". 

Virtually  all  of  the  operators  who  preferred  the  color 
display  also  commented  that  it  was  easier  to  pick  out  specific 
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.  The  numbers  of  officers  and  enlisted  personnel 
preferring  each  type  of  display. 


Di splay 

Monochrome  Color 


Personnel 

Off i cers 
Enl i sted 


7 

4 


5 

8 


Total 


1 1 


13 


targets  or  objects,  but  for  different  reasons.  Some  said  objects 
seemed  more  distinct  and  more  separable.  Others  said  color  coded 
objects  were  easier  to  recognize  and  identify.  One  operator 
stated  that  the  color  display  provided  better  coordination 
between  the  situation  display  and  the  tabular  display. 

Thus,  both  those  preferring  the  monochrome  display  and  those 
preferring  the  color  display  found  reasons  that  the  display  of 
their  preference  made  it  easier  to  pick  out  specific  targets  or 
objects.  There  are  some  additional  comments  made  by  those 
preferring  the  monochrome  display,  however,  that  temper  the 
strength  of  their  preference  and  argue  for  some  use  of  color. 

They  said  that,  even  though  they  preferred  monochrome,  color  was 
beneficial  for  some  things.  Some  of  the  comments  were  that  color 
was  good  for  the  background  volumes,  for  color  coding  the 
hostiles,  for  the  Engaged  Data  Table,  for  high  track  loads,  and 
for  enhancing  the  contrast  of  the  objects.  One  operator  said 
that  color  was  helpful  although  overused  and  another  said  that 
"maybe  (color)  Just  takes  getting  used  to." 

In  addition  to  being  asked  to  comment  on  why  they  preferred 
the  display  they  did,  the  operators  were  asked,  if  colors  were  to 
be  used,  which  colors  they  would  keep  the  same  and  which  ones 
they  would  change.  Here  the  operators  preferring  monochrome  and 
those  preferring  color  generally  made  the  same  type  of  comments. 
The  most  frequent  comment  was  that  the  number  of  different  colors 
should  be  kept  to  a  minimum.  Some  operators  suggested  using  only 
colored  outlines  on  the  volumes  rather  than  color-fill.  Also, 
many  commented  on  the  difficulty  they  had  reading  the 
alphanumeric  labels,  that  is,  the  target  numbers  and  volume 
labels.  The  colors  seemed  to  mix  together  making  them  hard  to 
see.  The  masked  terrain  and  range  rings  colors  specifically 
needed  to  be  made  brighter  or  darker.  Several  of  the  operators 
thought  that  color  could  be  used  to  advantage  on  the  Alert 
Message  Line  and  on  other  tables  such  as  the  Track  Amp  Table  and 
some  of  the  fault  or  status  tables.  Some  operators  suggested 
changing  the  friendly  color  from  blue  to  green  or  white.  All 
operators  agreed  that  red  was  the  best  color  for  hostiles. 
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5^0 _ QQDQius  j_ons_and_Recgmmenda  t  igns^ 

QveralJ _ LeyeJ _ Qf_QBeratgr_Perfgrmance.  Operator 

performance  was  below  or  deviated  from  system  performance  on  all 
measures  even  though  the  number  of  tracks  was  lower  than  that 
which  actually  could  be  expected  in  an  air  battle.  Two  reasons 
why  this  result  might  be  an  artifact  of  the  exper i ment--the  lack 
of  recent  operator  training  and  the  disparity  between  some 
aspects  of  the  Color-PTOS  and  the  tactical  Patriot  system--were 
mentioned  in  the  beginning  of  the  Results  and  Discussion  section. 
A  third  possible  reason  for  the  difference  between  operator  and 
system  performance  levels  is  not  an  artifact  of  the  experiment. 
The  reason  is  that,  even  at  the  track  loads  seen  here,  the 
display  is  cluttered,  the  operators  are  overloaded  with 
information  and,  therefore,  their  performance  is  degraded.  In 
the  Introduction,  this  situation  was  identified  as  a  driver  of 
this  research.  Clearly,  display  clutter  and  information  overload 
were  correctly  identified  as  problems. 

5^2  Effects_gf_Cgigr_gn_Summary_Perfgrmance.  As  measured 
at  the  mission  level,  the  color  display,  in  the  words  of  one  of 
the  test  operators,  "Just  takes  getting  used  to."  At  first,  the 
operators  were  'thrown  off  by  the  unfamiliar  color  display,  but 
after  just  an  hour  of  practice  on  the  simulation  system,  their 
performance  with  the  color  display  was  at  the  same  level  as  that 
with  the  monochrome  display.  Ulhat  is  not  known  from  this  study 
is  whether  further  practice  would  have  resulted  in  better 
performance  with  the  color  display  compared  to  that  with  the 
monochrome  one. 

——3  — f f — QQiQC_QD_l9sk_Perf grmance .  As  measured  by 
the  more  specific  TPMs,  the  color  display  did  not  help  or  hurt 
the  t i me  1 iness  of  engagements,  and  it  did  not  increase  or 
decrease  the  number  or  duration  of  the  various  operator 
activities.  In  other  words,  the  use  of  color  did  not  produce  a 
faster--or  s 1 ower--operator .  However,  there  is  evidence  that  the 
use  of  color  contributed  to  an  operator's  ef f i c i ency--the 
decrease  in  the  percentage  of  early  engagements  and  the  taking 
advantage  of  the  automatic  hooking  feature.  Although  it  is  not 
entirely  clear  how  the  "building  blocks"  of  task  performance 
combine  to  construct  mission  level  performance,  the  importance  of 
the  specific  tasks  should  not  be  underestimated.  An  increase  in 
the  efficiency  of  information  processing  at  the  task  level  could 
eventually  translate  into  an  improvement  at  the  mission  level, 
for  example,  after  extended  operations. 

_QQiQC_QD_§yb ject j_ye_ygrkigad .  As  measured 
subjectively,  the  workload  involved  in  performing  the  Uleapons 
Controller  task  decreased  over  time  with  the  color  display  but 
not  with  the  monochrome  display.  Also,  the  reduction  in  workload 
was  greater  the  heavier  the  track  load.  Note  that  practice  alone 
did  not  make  the  task  seem  easier.  It  took  the  combination  of 
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both  practice  and  the  color  display  to  reduce  the  subjective 
assessment  of  workload. 

fl£l§_Qf _IC§iDiD9 •  Underlying  all  the  results  is  the 
constant  factor  of  training  or  practice.  The  color  display 
significantly  impaired  mission  level  performance  until  the 
operators  were  more  practiced.  Also,  both  the  increased 
efficiency  of  information  processing  and  the  reduction  of 
subjective  workload  were  found  only  after  sufficient  practice. 
Perhaps  the  strongest  conclusion  to  be  drawn  from  these  results, 
then,  is  that,  with  any  major  display  modification,  training  and 
practice  are  critical. 

5^6  Ef  feet  s_of_Coior_on_SubJectiye_Ra  tings  j._DisBiay 
EC§f§C§Dce_and_Comments .  Subjective  measurement  of  eye  and 
vision  comfort,  the  appearance  of  the  displayed  image,  and 
overall  ease  of  viewing  revealed  that  the  test  operators  did  not 
feel  there  were  any  differences  between  the  two  displays  except 
in  what  was  salient  about  the  two  displays.  The  individual 
object  was  more  salient  on  the  monochrome  display  and  the  general 
appearance  of  the  display  objects  was  more  salient  on  the  color 
display.  And  although  the  color  display  was  favored  by  a  slim 
margin  over  the  monochrome  display,  stronger  support  for  the  use 
of  color  in  the  Patriot  display  comes  from  the  operator  comments. 
The  thrust  of  the  comments  is  that  fewer  colors  should  be  used 
either  because  the  current  colors  did  not  have  sufficient 
contrast  or  because  they  were  used  to  code  too  many  different 
levels  of  information.  In  other  words,  there  were  both 
perceptual  problems  and  problems  with  the  ways  in  which  color  was 
used,  that  is,  the  coding,  decluttering  and  highlighting.  Both 
problems  can  be  addressed  by  reducing  the  number  of  colors. 

5iZ  5§9Q!!!!D§Dd§liQDS .  The  most  specific  recommendation  to 
come  out  of  this  study  is  that,  if  color  is  to  be  used  in  the 
Patriot  display,  use  even  fewer  colors.  Using  fewer  colors  will 
make  it  simpler  to  ensure  sufficient  color  contrast  and,  at  the 
same  time,  restrict  the  number  of  ways  in  which  color  can  be 
used.  A  final  recommendation  on  the  number  and  combination  of 
colors  would,  ideally,  involve  prioritizing  all  of  the  possible 
uses  of  color  and  asking  Patriot  operators  and  subject  matter 
experts  to  comment  on  various  implementations.  For  example,  the 
use  of  different  colors  for  different  volume  types  could  turn  out 
to  be  a  low  priority.  If  all  volumes  were  the  same  color, 
individual  volumes  would  retain  the  distinct,  unified  appearance 
seen  here,  but  multiple  color  coding  levels  would  not  be  used. 
Color  codes  per  se  could  be  reserved  for  the  tracks,  friendly  - 
blue,  unknown  -  yellow  and  hostile  -  red.  Alternatively,  the 
background  volumes  could  be  color  coded  and  only  specific  tracks 
would  then  be  hi ghl i ghted--the  most  threatening  tracks,  the  track 
at  the  top  of  the  TBEQ,  the  track  being  referenced  by  the  Alert 
Message  Line,  or  the  hooked  track.  Also,  the  solid  color-fill 
for  the  search  and  track  sectors  could  be  eliminated  which  would 
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reduce  the  number  of  background  colors  against  which  objects  had 
to  be  contrasted.  Further,  the  alphanumeric  labels  could  be  all 
in  white  for  maximum  visibility.  Color  highlighting  could  also 
be  used  on  all  the  main  tables,  but  limited  to  a  single  alerting 
color  rather  than  multiple  colors. 

Finally,  the  more  general  recommendation  to  come  out  of  this 
study  has  to  do  with  reconciling  the  meager  performance 
enhancements  obtained  with  the  color  display  with  the  striking, 
obvious  differences  in  appearance  between  the  monochrome  and  the 
color  displays.  A  key  distinction  to  be  made  is  the  difference 
between  what  the  viewer  sees  and  what  the  user  sees.  The  viewer 
sees  the  display  in  it  overall  configuration,  whereas  the  user, 
being  oriented  to  the  task,  sees  the  specific  display  details. 
Thus,  this  particular  color  display  reduced  the  overall  sense  of 
workload,  but  it  did  not  enhance  performance.  The  two  questions 
remaining  are  (1)  whether  the  reduced  workload  brought  about  by 
the  overall  use  of  color  will  eventually  translate  into  enhanced 
performance  at  higher  track  loads  or  after  extended  operations, 
and  (2)  whether  color  can  be  used  to  enhance  performance  as  it 
depends  on  specific  display  details. 
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APPENDIX  A 

SUBJECTIVE  WORKLOAD  ASSESSMENT  TECHNIQUE  (SWAT) 
MATERIALS  AND  EXAMPLES 


A-1 


Ei9yC§_^ii*  example  of  one  of  the  27  SWAT  sorting  cards 

showing  Time  Load  level  1,  Mental  Effort  Load  level  3,  and 
Psychological  Stress  Load  level  2. 


TIME  LOAD  -  Often  have  spare  time.  Interruption 
or  overlap  among  activities  occur 
infrequently  or  not  at  all. 

MENTAL  EFFORT  LOAD  -  Extensive  mental  effort  and 

concentration  are  necessary.  Very 
complex  activity  requiring  total 
attent i on. 

PSYCHOLOGICAL  STRESS  LOAD  -  Moderate  stress  due  to  confusion, 

frustration,  or  anxiety  noticeably 
adds  to  workload.  Significant 
compensation  is  required  to  maintain 
adequate  performance. 
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I§bXe_Aj.i.  The  SWAT  scale  obtained  for  the  test  operators  in 
this  experiment. 


- Dimensions  of  Workload - 

Time  Mental  Psychological  Scale  Values 

Effort  Stress 

- Levels - 


0.0 
18. g 

38.4 
12.9 

31.7 

51.2 

26.7 

45.6 
65.  1 

15.4 

34.3 

53.8 

28.3 

47.1 

66.6 

42. 1 
61.0 

80.5 

34.9 

53.8 

73.3 

47.8 

66.6 

86.1 
61  .6 
80.5 

100.0 


Note.  A  score  of  0  represents  low  workload  and  100  represents 
high  workload.  The  ratings  on  the  three  dimensions  of  workload 
are  translated  to  a  single  score  using  this  scale.  For  example, 
if  an  individual  rated  a  task  1,  3,  and  2  on  the  three 
dimensions,  that  translated  to  a  single  score  of  45.6. 
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Eisyrg-Aig-  The  instructions  and  the  SWAT  rating  scales  used  for 
rating  individual  scenarios. 

Please  rate  the  scenario  you  Just  completed  on  each  of  the  three 
dimensions  of  workload.  Circle  the  appropriate  number. 

Time  Load 


1 

2 

3 

Often  have  spare 
time.  Interruptions 
or  overlap  among 
activities  occur 
infrequently  or 
not  at  all. 

Occasionally  have  spare 
time.  Interruptions  or 
overlap  among  activities 
are  very  frequent. 

Almost  never  have 
spare  time.  Inter¬ 
ruptions  or  over¬ 
lap  among  activi¬ 
ties  are  very 
frequent,  or  occur 
al 1  the  time. 

Mental  Effort  Load 

1 

2 

3 

Very  1 i tt 1 e  con¬ 
scious  mental 
effort  or  concen¬ 
tration  required. 
Activity  is  almost 
automatic,  requiring 
little  or  no 
attent i on. 

Moderate  conscious  mental 
effort  or  concentration 
required.  Complexity  of 
activity  is  moderately 
high  due  to  uncertainty, 
unpredictability,  or 
unfamiliarity.  Consid¬ 
erable  attention 
required. 

Extensive  mental 
effort  and  concen¬ 
tration  are  neces¬ 
sary.  Very  com¬ 
plex  activity 
requiring  total 
attent i on. 

Psychological  Stress 

Load 

1 

2 

3 

Little  confusion, 
risk,  frustration, 
or  anxiety  exists 
and  can  be  eas i 1 y 
accommodated. 

Moderate  stress  due  to 
confusion,  frustration, 
or  anxiety  noticeably 
adds  to  workload.  Sig¬ 
nificant  compensation 
is  required  to  maintain 
adequate  performance. 

High  to  very 
intense  stress  due 
to  confusion, 
frustration,  or 
anxiety.  High  to 
extreme  determina¬ 
tion  and  self- 

control  required. 
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APPENDIX  B 

SUBJECTIVE  OPINION  RATING  MATERIALS 


B-1 


Ei9yC§_Biii  The  rating  scales  used  for  the  subjective  ratings  of 
the  two  dimensions  of  eye  comfort,  the  two  dimensions  of  viewing 
comfort,  the  five  dimensions  of  image  appearance,  and  overall 
viewabi 1 i ty . 


You  have  now  completed  six  scenarios,  all  with  the  monochrome 
(color)  display.  Think  back  over  all  six  of  the  scenarios  as  you 
make  the  following  ratings.  Circle  the  appropriate  number 


Use  these  two  rating  scales  to  describe  how  your  eyes  felt  while 
you  worked  on  the  six  scenarios.  Circle  the  appropriate  number. 


1 

_2 _ 

3 

4 

5 

t  i  red 

rested 

1 

_2 _ 

3 

4 

5 

burni ng 

norma  1 

Use  these  two  rating  scales  to  describe 
were  seeing)  while  you  worked  on  the  six 

your  vision 
scenarios. 

(how  wel 1  you 

1 

_2... 

3 

4 

5 

blurry 

c  1  ear 

1 

_2 _ 

3 

4 

5 

changing 

consi stent 

Use  these  five 
display  objects 

rat i ng 
in  the 

scales  to  describe 
six  scenarios. 

the 

appearance  of  the 

1 

_2 _ 

3 

4 

5 

f 1 i cker i ng 

stab  1 e 

1 

2 _ 

3 

4 

5 

low  CO  1 or/br i ghtness 
contrast 

high  col 

1  or/brightness 
contrast 

1 

_2 _ 

3 

4 

5 

fuzzy 

c  1  ear 

1 

_S _ 

3 

4 

5 

hard  to  read/ 
i 1  legible 

easy  to  read/ 
legible 

1 

_2 _ 

3 

4 

5 

confused  with 
other  objects 

separate  &  distinct 
from  other  objects 

Overall,  how  easy  was  this  display  to  view? 
this  display  on  ease-of-vi ewi ng. 

In  other  words,  rate 

1 

_2 _ 

3 

4 

5 

hard  to  view  easy  to  view 
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APPENDIX  C 

INTERCORRELATION  MATRICES  OF 

THE  RATINGS  OF  EYE  COMFORT,  VIEWING  COMFORT,  IMAGE  APPEARANCE, 

AND  OVERALL  EASE  OF  VIEWING 
FOR  THE  MONOCHROME  AND  COLOR  DISPLAYS 
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Table  C.l.  Correlation  matrix  for  the  ratings  of  the  monochrane  display  on  the  dimensions  of  eye  cmfort 
vision  comfort.  Image  apperance  and  of  overall  ease  of  viewing.  Correlations  marked  with  an  asterisk  are 
significant  at  p  <  .05.  EC  =  eye  comfort,  VC  =  vision  comfort,  lAP  =  Image  appearance,  and  BOV  =  ease  of 
viewing.  Ihe  words  under  the  variable  labels  are  the  endpoints  of  the  rating  scales. 
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APPENDIX  D 
OPERATOR  COMMENTS 


The  test  operators  made  comments  in  response  to  the 
following  questions.  (Comments  of  pilot  test  operators  are  also 
included  here. ) 

1.  You  have  now  completed  all  12  scenarios.  The  first  six 
were  with  the  monochrome  (color)  display;  the  second  six  were 
with  the  color  (monochrome)  display.  Thinking  back  about  both 
displays,  overall,  which  display  did  you  prefer? 

2.  Please  give  the  reasons  why  you  preferred  the  display 
you  did. 


3.  If  the  Patriot  weapon  system  were  to  have  a  color 
display  like  the  one  you  saw  here  today,  what  changes  would  you 
make,  if  any?  For  example,  would  you  change  the  colors?  Ulould 
you  use  the  colors  differently?  Would  you  add  color  to  any  other 
part  of  the  display? 

4.  Would  you  change  the  color  used  for  these  specific 
display  objects?  If  so,  how?  (Followed  by  a  list  of  display 
objects. ) 


D-1 


PILOT  OPERATOR  #1 
RANK:  ILT 

MDS:  14E 

PREFERENCE:  COLOR 

REASONS:  The  different  colors  provided  an  easier  identification  of 

areas/targets.  Although  the  symbology  of  enemy/frlends  are  distinct 
the  color  made  it  more  apparent  of  viho  was  \diere.  I  feel  the  colors 
are  softer  on  the  eyes  than  the  monochrome  was.  Color  makes  the 
scenario  seem  more  like  a  game  versus  monochrane,  it  is  still  more 
pleasant  to  look  at.  I  had  no  trouble  identifying  the  symbology. 

The  change  to  color  is  a  matter  of  getting  accustcxned  to  -  great 
work. 

CHANGES:  I  feel  the  colors  selected  worked  fine,  I  do  reccximend  keeping  it  to 
a  minimum. 

SPECIFIC  CHANGES: 

MASKED  TERRAIN:  Could  be  made  a  little  more  Intense  in  color. 

PILOT  OPERATOR  #2 
RANK:  2LT 

MOS:  14E 

PREFERENCE:  MONOCHROME 

REASONS:  I  prefer  the  monochrome  because  it  was  not  as  bright  as  the  color. 

The  color  screen  was  easier  to  read,  but  it  was  very  tiring  because 
the  colors  used  were  too  bri^t  and  forceful — you  should  use  dull 
colors . 

CHAMJES:  Yes,  I  would  change  the  colors  and  like  I  said  I  would  change  than 
to  dull  colors-more  green-I  would  also  make  the  Prohibited  and 
Restricted  areas  different  colors.  All  the  numbers  labeling  the 
areas  should  be  black  because  with  the  colors  used  now  they  tend  to 
inter-mix  making  than  Impossible  to  read. 

SPECIFIC  CHANGES: 

MASKED  TERRAIN:  Black 
WEAPONS  CONTROL  VOLUME-HOLD:  Light  gray 
WEAPONS  CONTROL  VOLUME-FREE:  Medium  gray 
WEAPONS  CONTROL  VOLUME-TIGHT:  Dark  gray 
RESTRICTED  VOLUME:  Light  brown 
PROHIBITED  VOLUME:  Dark  brown 

PILOT  OPERATOR  #3 
RANK:  ILT 

MOS:  14E 

PREFERENCE:  MONOCHROME 

REASONS:  The  color  display  tended  to  make  up  a  3  dimensional  type  atmosphere, 
which  I  was  not  used  to  working  with.  The  monochrome  always 
appeared  flat,  like  a  radar  screen.  The  only  advantage  I  see  to 
color  is  the  contrast  ability,  but  I  don't  have  any  problon  knowing 
what  the  (areas,  volumes,  etc.)  feature  I'm  looking  at  is  anyway.  I 
definitely  like  using  monochrane  because  it  feels  more  like  what  it 
le,  a  radar  screen. 

CHANGES:  I  would  not  switch,  but  if  I  did,  I  would  use  as  few  different 
colors  as  possible,  and  make  them  contrast  as  much  as  possible. 

Stay  away  from  the  3-D  effect.  As  for  colors  used  here  in  this 
test,  there  were  too  many  different  ones  without  enou^  contrast. 
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PILOT  OPERATOR  if5 
RANK:  2LT 

MOS:  14E 

PREFERENCE:  COLOR 

REASONS:  I  preferred  the  color  display.  The  reasons  are  that  things  were 
much  more  distinguishable.  I  could  tell  exactly  what  each  TGT  was 
and  where  It  was.  Tbe  corridors  and  other  display  stood  out  very 
well,  It  was  easy  to  see  and  read.  I  did  not  have  to  strain.  It 
was  not  a  strain  on  my  eyes.  A  few  colors  could  be  changed  but  It 
was  very  good,  especially  the  Data  Tabs. 

CHANGES:  I  would  change  the  color  of  the  volumes  because  they  look  too  dull. 

The  Free  volumes  could  be  red  as  are  the  hostlles  because  this  notes 
danger.  The  Hold  &  Tight  could  be  blue  for  possible  friendly  or 
targets  not  to  be  fired  on.  The  Sector  could  be  darker  gray  to 
bring  out  the  other  colors.  If  the  display  Is  a  little  darker  It  Is 
not  such  a  strain.  The  Prohibited  and  Restricted  could  also  be  a 
red  for  danger,  maybe  a  darker  red.  I  liked  the  Tabs  black. 

SPECIFIC  CHANGES: 

PEBA:  Change  to  black  as  It’s  better  visible. 

SEARCH  SECTOR:  Make  It  darker 
TRACK  SECTOR:  Darker 

RANGE  RINGS:  Change  to  black.  They  show  up  better 

WEAPONS  CONTROL  VOLUME-HOLD:  Blue 

WEAPONS  CONTROL  VOLUME-FREE:  Red 

WEAPONS  CONTROL  VOLUME-TIGHT:  Blue 

RESTRICTED  VOLUME:  Red 

PROHIBITED  VOLUME:  Red 

PILOT  OPERATOR  #6 
RANK:  SSG 

MOS:  2ilT 

PREFERENCE:  COLOR 

REASONS:  Several  Instances  It  was  much  easier  to  see  the  targets  due  to  their 
color  designator.  Video  display  was  sharp  and  clear.  Had  no 
problems  with  the  color  display.  Much  Preferred. 

CHANGES:  At  this  time  I  would  not  change  any  colors. 

SPECIFIC  CHANGES: 

ALERT  MESSAGE  LINE:  Change  the  alert  line  color  to  RED 

OPERATOR  #1 
RANK:  SSG 

MOS:  2ilT 

PREFERENCE:  MONOCHROME 

REASONS:  I  prefer  monochrane  because  (1)  contrast  of  targets  and  background 
was  better  (2)  less  eye  strain. 

CHANGES:  I  would  delete  sector  coverage  color  or  change  It  In  a  way  that  all 
TGT's  will  stand  out  better.  I  would  change  the  color  of  the  alerts 
(they  do  not  stand  out) .  It  may  cause  less  eye  strain  If  only  the 
outline  of  each  shape  was  In  color.  Instead  of  the  Information  In 
"ENG  TAB"  turning  red  when  "RT"  reaches  0.  RT  should  turn  red  vhen 
"TLL"  reaches  10  or  less. 

SPECIFIC  CHANGES: 

SEARCH  SECTOR:  No  color 
TRACK  SECTOR:  No  color 

ALERT  MESSAGE  LINE:  Red  or  any  color  that  will  stand  out  better  In  white. 
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OPERATOR  #2 
RANK:  SSG 

MOS:  24T 

PREFERENCE :  MONOCHROME 

REASONS:  I  don't  really  care  for  the  full  color  picture.  I  prefer  the  lines 
colored  as  a  reference  or  guide.  I  prefer  the  monochrcxne  also 
because  I  like  as  little  brightness  as  possible  frcxn  the  scope 
because  it  is  easier  on  the  eyes.  The  engage  data  Is  also  easier  to 
read,  but  I  would  like  to  see  targets  that  have  zero  RT  in  a 
different  color. 

CHANGES:  I  would  use  colored  outlines  and  I  would  add  some  colors  for  the 

different  sector  bounds.  Staying  with  the  Idea  of  only  coloring  the 
outlines.  I  also  like  a  dark  background  behind  everything. 

SPECIFIC  CHANGES: 

BACKGROUND:  Black 

ENGAGED  DATA  TAB:  I  like  the  color  change 

OPERATOR  #3 
RANK:  ILT 

MOS:  li|E 

PREFERENCE:  MONOCHROME 

REASONS:  I  liked  the  dark  and  light  contrast.  It  v^s  easier  to  concentrate 
on  the  targets  In  monochrcxne  because  they  were  a  little  brl^ter. 
However,  the  monochrcme  does  have  Its  drawbacks:  (1)  Prohibited  & 
Restricted  areas  were  easy  to  ignore.  (2)  When  there  were  a  lot  of 
tracks  It  was  more  confusing  than  the  color  display.  Monochrcxne  is 
harder  on  the  eyes.  I  would  like  to  suggest  another  alternative.  I 
think  if  the  background  was  black  like  the  monochrome,  but  the 
outlines  done  in  different  colors,  then  you  would  have  an  optimal 
syston. 

CHANGES:  I  think  there  should  be  a  ccxnbination  of  the  two  displays.  You 

should  keep  the  dark  background  of  the  monochrome  but  have  different 
colored  outlines.  Free,  Hold  and  Tight  areas  should  be  In  different 
colors.  Prohibited  and  Restricted  areas  should  be  outlined  in 
different  colors.  Range  Rings  should  be  different  because  they  are 
very  hard  to  see.  Masked  Terrain  is  hard  to  see  too. 

SPECIFIC  CHANGES: 

SEARCH  SECTOR:  Gray 
TRACK  SECTOR:  Black 

MASKED  TERRAIN:  White-dashed  line.  *  If  you  change  the  Search  & 

Track  sectors,  then  leave  this  alone. 

WEAPONS  CONTROL  VOLUME-HOLD:  Dark  orange  solid  outline 
WEAPONS  CONTROL  VOLUME-FREE:  Yellow  solid  outline 
WEAPONS  CONTROL  VOLUME-TIGHT:  Light  orange  solid  outline 
RESTRICTED  VOLUME:  Pink  outline-dashed 
PROHIBITED  VOLUME:  Red  outline-dashed 
DEPENDED  AREA/POINTS:  Solid 
SAFE  PASSAGE  CORRIDOR:  Outline  only 
BACKGROUNG:  Black 

ENGAGED  DATA  TAB-PARTICULARLY  THE  COLOR  WHEN  RT  GOES  TO  0:  Stay  red; 
tab  should  be  reversed. 

OTHER  TABS:  Track  AMP  data-RT  should  turn  red  when  It  goes  to  zero. 

ALERT  MESSAGE  LINE:  Li^t  gray  or  black 
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SPECIFIC  CHANGES: 

FRIENDLY  TRACKS:  Green 
UNKNOWN  TRACKS:  Yellow 
HOSTILE  TRACKS:  Red 
UJIPS,  PIPS,  TRAILS:  White 
MISSILES:  Red 

PEBA:  Blue-separate  with  dotted  lines 
SEARCH  SECTOR:  Blue-separate  with  dotted  lines 
TRACK  SECTOR:  Blue-separate  with  dotted  lines 
RANGE  RINGS:  Leave  off-black  dotted  line 
MASKED  TERRAIN;  Leave  off-black  dotted  line 
GENERAL  POIMTS :  Green 
CRG:  Green 

BATTALION  FLAG:  Green 

CURSOR;  White 

NORTH  SYMBOL:  White 

WEAPONS  CONTROL  VOLUME-HOLD:  Green 

WEAPONS  CONTROL  VOLUME-FREE;  Red 

WEAPONS  CONTROL  VOLUME-TIGHT:  Yellow 

RESTRICTED  VOLUME:  White-all  dotted  lines 

PROHIBITED  VOLUME:  White-all  dotted  lines 

ffiPENffiD  AREA/POINTS:  White-all  dotted  lines 

SAFE  PASSAGE  CORRIDOR:  White-all  dotted  lines 

BACKGROUND;  Blue 

ENGAGED  DATA  TAB;  ?  Not  in  color  gray 
OTHER  TABS:  ?  Not  in  color  gray 
ALERT  MESSAGE  LINE:  ?  Not  in  color  gray 

PILOT  OPERATOR  #4 
RANK:  2LT 

MOS:  14E 

PREFERENCE ;  MONOCHROME 

REASONS:  The  reason  I  chose  monochrome  is  because  to  me  it  seemed  clearer  and 
more  distinct.  Uie  different  shades  in  the  monochrone  allows  the 
operator  to  distinguish  one  object  from  the  other.  Also  it  keeps 
the  systan  simpler. 

CHANGES:  Yes,  I  would  change  some  of  the  colors  because  they  seaned  to  be  too 
similar  vdilch  made  reading  or  distinguishing  them  difficult  at 
times.  Please  see  the  following  sheets  for  how  I  would  use  the 
colors  differently,  and  no,  I  would  not  add  any  more  colors  to  the 
screen. 

SPECIFIC  CHANGES: 

FRIENDLY  TRACKS;  Yes,  this  should  be  green. 

UNKNOWN  TRACKS:  Yes,  this  should  be  blue. 

RANGE  RINGS:  Yes,  these  colors  seaned  to  meld  into  the  background  color. 
They  ought  to  be  darker. 

RESTRICTED  VOLUME:  Yes,  I  could  not  see  this  color  behind  the  color  for 
Weapon  Control  Volumes. 

BACKGROUND:  Yes,  the  present  color  allows  too  many  other  colors  to  meld 
with  it. 

ENGAGED  DATA  TAB:  Yes,  the  white  TAB  background  was  too  bright. 
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OPERATOR 
RANK:  2LT 

MOS:  14E 

PREFERENCE:  MONOCHROME 

REASONS:  Monochrome  was  much  easier  to  read.  The  color  display  was  Just  too 
much  action  for  the  eyes.  I  could  rarely  read  the  track  numbers  in 
the  color  display.  When  a  red  (enany)  target  entered  an  orange 
(weapon  control)  it  was  bothersone  and  difficult  to  read.  The 
inside  of  an  ECS  is  very  dark  and  black  on  white  scenarios  are 
better.  I  hope  they  stay  with  monochrane.  Color  is  terrible, 
confusing  and  ranlnds  me  of  an  arcade  game. 

CHANGES:  Because  I  am  opposed  the  only  color  change  I  could  reccinmend  would 
be  brighter  track  numbers .  Harder  colored  reds  and  blues ! !  Also 
the  unknown  yellow  will  scmetlmes  mesh  into  a  blue  Friendly  and  be 
very  difficult  to  distinguish.  The  safe  passage  corridors  are  a 
good  Color  but  the  Restricted  Volumes  and  Prohibited  Volumes  should 
be  a  bit  brighter.. .stay  with  monochrane. 

SPECIFIC  CHANGES: 

FRIENDLY  TRACKS:  A  bit  darker  would  be  nice 

UNKNOWN  TRACKS:  Perhaps  green,  yellow  is  very  Irritating  and  hard  to 
see. 

HOSTILE  TRACKS:  Must  stay  red! ! ! 

PEBA:  Not  really  noticeable,  don't  really  refer  to  it  much. 

TRACK  SECTOR:  No  problem,  could  be  a  bit  brighter  line.  I  sonetimes 
forgot  it  was  there. 

RANGE  RINGS:  Rarely  use  this  anyway  in  the  ECS,  makes  for  too  much 
clutter. 

GENERAL  POINTS:  Good,  not  used  much  but  easy  to  find  and  reference 
CRG:  Green  really  stands  out.  Good  color. 

BATTALION  FLAG:  Green  is  good,  easy  to  distinguish. 

RESTRICTED  VOLUME:  Pinkish  or  orangish.  Color  is  too  light  and 
irritating. 

PROHIBITED  VOLUME:  Same  as  above,  difficult  to  read  track  numbers  at 
times. 

DEFENDED  AREA/POINTS:  Easy  to  see  boxes,  sonetimes  defended  points 
seem  too  small. 

SAFE  PASSAGE  CORRIDOR:  Blue  is  excellent  color. 

BACKGROUND:  Good  in  both  monochrane  and  color.  Prefer  darkest  background 
possible. 

ENGAGED  DATA  TAB:  Liked  how  it  turned  to  red  when  RT  reached  0,  very 
noticeable . 

ALERT  MESSAGE  LINE:  A  bit  annoying  when  I  kept  getting  alerts  on  already 
engaged  targets  but  the  lines  thanselves  were  no  problem  to  read. 

OPERATOR  #5 
RANK:  2LT 

MOS:  14E 

PREFERENCE:  MONOCHRCME 

REASONS:  With  monochrane  there  were  less  distractions  to  filter  through  than 
with  color.  The  blinking  and  varying  brightness  were  enough  to 
distinguish  tracks  from  each  other  when  needed.  The  color  was 
helpful  but  was  overused  and  would  be  much  more  useful  in  limited 
situations . 
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CHANGES:  Eton’t  make  everything  different  colors.  The  colors  don't  make 
things  stand-out  If  everything  has  Its  own  color.  If  only  seme 
things  such  as  engagable  tracks  and  tracks  with  alerts  on  than  were 
colored  different.  It  would  make  them  stand  out  Immensely  and  be  a 
big  help  to  the  operator.  Use  color  around  outside  of  volumes, 
corridors,  etc.,  rather  than  filling  In  solids. 

SPECIFIC  CHANGES: 

HOSTILE  TRACKS:  Red  is  Great! 

TRACK  SECTOR:  Shading  makes  search  sector  3-D  instead  of  setting  off 
track  sector  for  search. 

RANGE  RINGS:  Change  to  sane  distinct  color.  They  are  hard  to  see  as  are. 
DEPENDED  AREA/POINTS:  Change  to  STAND-OUT  more 

ENGAGED  DATA  TAB-PARTICULARLY  THE  COLOR  WHEN  RT  GOES  TO  0:  Very  helpful 
as  is  setting  off  lines  with  red. 

OPERATOR  #6 
RANK:  2LT 

MOS:  lilE 

PREFERENCE :  MONOCHRCME 

REASONS:  The  monochrone  display  was  more  conducive  to  picking  out  particular 
(target)  display  than  the  color.  When  going  from  the  TBE  tab  to  the 
display  ray  eyes  would  require  a  great  deal  of  scanning  to  discern 
the  object  in  question.  In  all  the  relative  lack  of  color  in  the 
monochrome  display  proved  easier  to  work  with. 

CHANGES:  Color,  for  colors  sake,  falls  to  acconpllsh  the  objective  of  making 
certain  Important  objects  stand  out.  I  think  that  reducing  the 
number  of  objects  with  color  to  Just  those  with  a  high  priority 
would  be  more  effective.  (Example,  get  rid  of  color  In  the  Search 
and  Track  sector) .  If  everything  has  a  different  color  they  all 
tend  to  blend  and  nothing  Jumps  out.  If  Just  priority  or  hostile 
targets  were  a  different  color  I  believe  the  operator  would  benefit. 
SPECIFIC  CHANGES: 

FRIENDLY  TRACKS:  Normal  green 

UNKNOWN  TRACKS:  Normal  green 

HOSTILE  TRACKS:  Red 

TRACK  MODIFIERS:  Normal  green 

LNIPS,  PIPS,  TRAILS:  Normal  green 

MISSILES:  Normal  green 

PEBA:  Normal  green 

SEARCH  SECTOR:  Normal  green 

TRACK  SECTOR:  Normal  green 

RANGE  RINGS:  Normal  gr«en 

MASKED  TERRAIN :  Some  stand-out  color 

GENERAL  POINTS:  Some  stand-out  color 

CRG:  Some  stand-out  color 

BATTALION  FLAG:  Sane  stand-out  color 

CURSOR:  Sane  stand-out  color 

NORTH  SYMBOL:  Sane  stand-out  color 

WEAPONS  CONTROL  VOLUME-HOLD:  Stand-out  color 

WEAPONS  CONTROL  VOLUME-FREE:  Stand-out  color 

WEAPONS  CONTROL  VOLUME-TIGHT:  Stand-out  color 

F5ESTRICTED  VOLUME :  Stand-out 

PROHIBITED  VOLUME:  Stand-out 

DEPENDED  AREA/POINTS:  Normal  color 
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SAFE  PASSAGE  CORRIDOR:  Normal  color 
BACKGROUND:  Normal  color 
ENGAGED  DATA  TAB:  Normal  color 
OTHER  TABS:  Normal  color 
ALERT  MESSAGE  LINE:  Normal  color 

OPERATOR  #7 
RANK:  SP4 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  Because  it  made  it  easier  for  me  to  Identify  targets  easier  and 
track  numbers . 

CHANGES:  I  would  change  the  color  of  the  Restricted  or  Prohibited  Volume 
because  they  are  too  close  together  in  color  and  makes  it  very 
difficult  to  tell  the  difference. 

SPECIFIC  CHANGES: 

FtESTRICTED  VOLUME:  To  a  darker  or  lighter  color 

PROHIBITED  VOLUME:  To  the  opposite  of  Restricted  Volume 

ENGAGED  DATA  TAB-PARTICULARLY  THE  COLOR  WHEN  RT  GOES  TO:  Liked  that 

OPERATOR  #8 
RANK:  PFC 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  My  reasons  for  picking  color  were  as  follows:  I  like  that  it  is 

made  easier  to  identify  tracks,  and  other  points  on  the  screen:  to 
sum  it  up  I  feel  the  color  screen  can  help  an  operator  find  things 
with  less  workloads. 

CHANGES:  Ihe  colors  that  I  had  seen  does  not  require  any  improvement,  one 
exception:  (Alert  Message  Line) 

SPECIFIC  CHANGES: 

OTHER  TABS:  These  could  have  entries  added  to  than  depending  on  urgency 
of  itans  on  these  tabs. 

ALERT  MESSAGE  LINE:  Could  use  a  more  active  color,  the  one  being  used 
was  too  passive. 

OPERATOR  #9 
RANK:  PFC 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  Clearer  more  distinct,  easy  to  separate  one  object  frcxn  another, 
color  showed  priority  on  important  objects. 

CHANGES:  P?ange  rings-bright  yellow. 

SPECIFIC  CHANGES: 

RANGE  RINGS:  Yellow 

OPERATOR  #10 
RANK:  SP4 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  I  liked  the  color  differentiation.  It  allowed  me  to  keep  ray  mind  on 
other  details  rather  than  looking  at  and  reaffirming  the  same  things 
over  and  over  because  I  forgot  (mainly  I.D.  areas).  Also,  it  was 
easy  on  the  eyes  as  far  as  contrast  and  brlgjitness  while,  at  the 
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same  time  remaining  very  much  legible.  The  color  display  also 
allowed  for  better  coordination  and  use  of  Tabular  display 
information  and,  also  too,  symbol  modifiers  were  very  sharp.  I 
believe  I  could  use  this  system  to  advantages  such  as  better 
coverage  with  same  equipnent  and  also,  operate  for  longer  time 
period  since  my  eyes  don’t  get  strained  as  much. 

CHANGES:  No.  For  the  most  part,  the  colors  used  vrtiere  used  good.  I  feel 
masked  areas  could  have  been  a  little  hi^er  in  contrast  due  to 
their  extrane  importance.  I  wouldn’t  mind  seeing  the  alert  line  in 
a  bright  shade  of  a  high  contrast  color  or  maybe  Just  certain  alerts 
come  up  in  color  (such  as  Prl  Eng)  in  bri^t-brlght  red,  while 
alerts  such  as  LS  LOW  MSL  could  ranaln  black. 

SPECIPICE  CHANGES: 

MASKED  TERRAIN:  Now  orange-would  change  to  a  brighter  shade  or  to  green. 

OTHER  TABS:  Maybe  label  one  color,  data  another. 

OPERATOR  #11 
RANK:  ILT 

MOS:  IHE 

PREFERENCE:  COLOR 

REASONS:  The  color  contrast  made  it  easier  to  read.  It  helps  the  operator  to 
make  decisions  faster. 

CHANGES:  I  would  change  the  blinking  missile  intercept  line. 

SPECIFIC  CHANGES:  None 

OTHER  TABS:  Maybe  Fault  Data  in  red  to  catch  the  eye. 

OPERATOR  #12 
RANK:  ILT 

MOS:  14E 

PREFERENCE:  COLOR 

REASONS:  (1)  Easy  to  distinguish  hostile,  unknown  and  friendly  tracks  (fran 
one  another).  (2)  Eye  strain  did  not  seem  to  be  as  evident.  (3) 
When  release  time  reached  zero,  on  a  specific  track,  TBE  data 
changed  to  red.  (4)  Ihe  ”X”  that  appeared  in  the  Track,  which  was 
being  referenced  by  an  alert.  (5)  Readability  of  character. 
CHANGES:  Range  Rings  -  use  a  color  which  is  easier  to  detect. 

SPECIFIC  CHANGES: 

Purple  is  hard  to  see,  without  straining  (change  to  something  brighter). 

OPERATOR  #13 
RANK:  CPT 

MOS:  14E 

PREFERENCE:  COLOR 

REASONS:  The  color  display  made  a  quick  distinction  between  volumes.  It  was 
much  easier  to  detemlne  what  was  what.  This  took  quite  a  mental 
workload  off  me. 

CHANGES:  NO,  the  colors  were  quite  good.  There  were  none  so  brlght/shaip 
that  it  distracted  me,  but  still  allowed  for  easy  distinction 
between  volumes.  I  would  not  use  colors  differently,  nor  would  I 
add  color  to  any  other  part. 

SPECIFIC  CHANGES: 

MISSILES:  Maybe  change  to  white 

CURSOR:  Not  so  ’’bright”  a  white.  It  often  distracted  me  so  I  would 

move  it  off  to  one  side. 
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WEAPCWS  CONTROL  VOLUME-FREE:  Maybe  a  red 

ENGAGED  DATA  TAB:  No  the  lines  going  to  red  when  RT  =  0  was  super! 

It  was  another  change  to  alert  the  operator  and  make  his  Job  easier. 

OPERATOR  nk 
RANK:  SSG 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  It's  alot  easier  to  recognize  objects  on  the  scope  and  alot  easier 
on  the  eyes. 

CHANGES;  I  believe  that  the  colors  are  very  nice  althou^  the  "pink"  for 
restircted  areas  seans  a  bit  too  bright. 

SPECIFIC  CHANGES: 

PIANGE  RINGS;  Yes,  make  than  a  bit  bri^ter. 

OPERATOR  #15 
RANK:  SPC 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  The  difference  in  the  color  contrast  will  help  the  operator  to 
distinguish  the  different  targets.  An  operator  who  is  at  the 
console  at  a  long  period  of  time  will  be  able  to  operate  with  less 
problens . 

CHANGES:  None 
SPECIFIC  CHANGES:  None 

OTHER  TABS:  Maybe  color  could  be  used  on  Track  Amp  Tab. 

OPERATOR  #16 
RANK:  ILT 

MOS:  14E 

PREFERENCE :  MONOCHRO/E 

REASONS:  I  preferred  the  monochrone  display  because  it  is  a  little  less 

confusing.  The  overlapping  of  color  made  it  hard  to  see  certain 
areas  of  concern  where  the  raonocolor  screen  had  less  color  that 
could  confuse  me.  It  might  Just  take  a  little  getting  used  to,  but 
today  I  was  more  comfortable  with  the  monochrome  screen. 

CHANGES:  I  would  suggest  using  less  color  than  the  color  CRT  had  today.  I 

felt  it  was  Interesting,  but  too  colorful.  I  think  the  use  of  color 
on  the  Patriot  CRT  is  a  good  idea,  but  not  everything  has  to  have  a 
color.  I  would  probably  like  Just  an  outlined  area  with  different 
colors  than  an  entire  section.  I  think  staying  with  the  mono 
(green)  screen  and  adding  Just  a  few  colors  to  onphaslze  important 
points  may  be  more  efficient. 

SPECIFIC  CHANGES: 

FRIENDLY  TRACKS:  White 

HOSTILE  TRACKS:  Red 

ENGAGED  DATA  TAB:  I  sonetlmes  used  it  to  decide  to  engage  v^en  it 

autcxnatlcally  went  red. 

OPERATOR  #17 
RANK:  SPC 

MOS:  24T 

PREFERENCE:  COLOR 

REASONS:  Ihe  color  was  easier  to  use  do  to  the  fact  I  was  able  to  distinguish 
different  objects  easier. 
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CHANGES:  I  would  have  to  change  some  of  the  colons. 

SPESIPIC  CHANGES: 

FRIENDLY  TRACKS:  Green 

UNKNOWN  TRACKS:  Yellow 

HOSTILE  TRACKS:  Red 

TRACK  MODIFIERS:  Same  as  Track 

LNIPS,  PIPS,  TRAILS:  Same  as  Track 

MISSILES:  White 

PEBA :  White 

ENGAGED  DATA  TAB:  Should  go  yellow  vdien  RT  is  down  to  10  and  red  v^en 
TLL  is  down  to  10. 

OTHER  TABS:  PP  Status,  Nogo=Red,  Degraded=Yellow.  Also  Operator  Access 
and  Missile  Inventory  Track  Amp,  Green/Red,  Garbled= Yellow. 

OPERATOR  #18 
RANK:  SP4 

MOS;  24T 

PREFERENCE:  COLOR 

FlEASONS;  It  is  easier  to  look  at  and  once  you  got  to  know  what  color 
represents  what  it  is  quicker  and  easier  to  find  what  you  are 
looking  for.  If  its  not  harder  to  troubleshoot  and  fix  than  the 
green  display  I  think  its  all  right. 

CHANGES:  Some  of  the  colors  kind  of  mixed  together  when  I  looked  at  thm. 

Seme  of  the  letters  got  covered  up  by  other  things.  Like  the  range 
rings  and  masked  terrain  dotted  lines  were  hard  for  me  to  see. 
SPECIFIC  CHANGES: 

PEBA :  Darker  lines 

FiANGE  RINGS:  Make  longer  dashes  or  darker 

MASKED  TERRAIN:  Make  longer  dashes  or  darker 

WEAPONS  CONTROL  VOLUME-HOLD,  WEAPONS  CONTROL  VOLUME-FREE,  WEAPONS 

CONTROL  VOLUME-TIGHT:  IT's  good  they  are  all  the  same  color. 

RESTRICTED  VOLUME,  PROHIBITED  VOLUME:  I  think  these  should  have  their 
own  colors. 

OPERATOR  #19 
RANK:  ILT 

MOS:  14E 

PREFERENCE:  COLOR 

REASONS:  Color  made  it  easier  to  differentiate  between  the  symbology.  White 
background  on  Tabular  Display  area  made  it  easier  to  read  and  helped 
the  alert  line  stick  out.  Overall  the  color  display  made  the 
console  less  monotonous  and  easier  to  read.  Didn't  care  for  the 
aspect  that  the  color  screen  doesn't  update  all  Tracks  and  Data 
simultaneously  though. 

CHANGES:  None,  except  Alert  Line  and  Track  when  Alerted  or  hooked. 

SPECIFIC  CHANGES: 

ALERT  MESSAGE  LINE:  Maybe  a  different  color  than  the  rest  of  the 
Tabular  Display. 

OPERATOR  #20 
RANK:  CPL 

MOS:  24T 

PREFERENCE:  MONOCHROME 
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REASONS:  The  monochrcme  display  is  very  simple  to  view  and  interpret,  lliere 
are  no  colors  to  conflict  (i.e..  The  prohibited  and  restricted 
volunes  and  the  weapon  control  volumes.  The  pink  and  orange  Just 
cancel  each  other  out).  The  raonochrone  is  easier  on  the  eyes. 
CHANGES:  I  wouldn't  worry  about  changing  a  thing  because  if  the  display  went 
to  color  like  I  saw  today,  I  would  walk  out  of  the  MOS. 

SPECIFIC  CHANGES:  None 

OPERATOR  #21 
RANK:  ILT 

MOS:  14E 

PREFERENCE:  COLOR 

REASONS:  The  color  display  gives  the  operator  a  better  perspective  of  the  Air 
Battle.  Helps  to  reduce  eye  fatigue.  Since  symbology  is  color 
coded,  adding  to  effectiveness  of  the  display. 

CHANGES:  Overall,  it  should  not  be  changed. 

SPECIFIC  CHANGES: 

RANGE  RINGS:  Hard  to  see  at  times.  Perhaps  a  darker  color. 

NORTH  SYMBOL:  Enlarge  the  North  Arrow. 

OPERATOR  #22 
RANK:  CPL 

MOS:  24T 

PREFERENCE:  MONOCHROME 

REASONS:  I  preferred  the  monochrome  Display 

a.  Objects  are  clearer  and  easier  to  see 

b.  Targets  are  more  distinct 

c.  I  found  the  color  display  to  be  distracting  with  all  the  different 
colors,  although  the  soft  pastel  colors  were  easy  on  the  eyes. 

d.  Ihe  monochrome  display  is  probably  a  great  deal  less  expensive. 
CHANGES:  If  I  had  to  choose  color  I  would  prefer  a  pale  blue  background  with 

all  volumes,  assets,  defended  areas  etc.  outlined  in  yellow  and  all 
targets  red.  The  red  target  was  the  easiest  to  see. 

SPECIFIC  CHANGES: 

FRIENDLY  TRACKS:  Red 
UNKNOWN  TRACKS:  Red 
HOSTILE  TRACKS:  Red 
IRACK  MODIFIERS:  Red 
LNIPS,  PIPS,  TRAILS:  Red 
MISSILES:  Red 
FEBA:  Yellow 

SEARCH  SECTOR:  Darker  blue 
TRACK  SECTOR:  Pale  blue 
RANGE  RINGS:  Yellow 
MASKED  TERRAIN:  Yellow 
GENERAL  POINTS:  Yellow 
CURSOR:  Yellow 

WEAPONS  CONTROL  VOLUME-HOLD:  Yellow 
WEAPONS  CONTROL  VOLUME-FREE:  Yellow 
WEAPONS  CONTROL  VOLUME-TIGHT:  Yellow 
RESTRICTED  VOLUME:  Yellow 
PROHIBITED  VOLUME:  Yellow 
DEFENDED  AREA/POINTS:  Yellow 
SAFE  PASSAGE  CORRIDOR:  Yellow 
BACKGROUND:  Black 

ENGAGED  DATA  TAB:  White  with  black  lettering  at  all  times. 


OPERATOR  §23 
RANK:  2LT 

PCS:  14E 

PREFERENCE:  MONOCHROME 

REASONS:  It  vfas  easier  to  pickout  separate  targets.  I  was  used  to  seeing 

display  this  way.  I  did  not  have  to  recognize  a  color  code  as  well 
as  a  shape  code. 

CHANGES:  I  would  use  only  outlined  colors  for  the  background,  and  the  thing 

we  need  to  see  at  a  scan.  In  solid  shades  of  color.  WCV-outllne,  ID 
volumes,  assets,  tracks-solld . 

SPECIFIC  CHANGES:  None 

OTHER  TABS:  Red  for  Negative  IFF  return  -  or  ID  change  from  U  to  H. 

First,  they  are  very  hard  to  see  at  a  glance.  If  the  rest  of  the  display 
was  not  blinking,  maybe  my  eyes  could  find  the  faster  blink  easier. 

Second,  the  X  and  0  symbol  did  not  add  or  subtract  fran  display.  It  Is 
the  blinking  that  was  of  note. 

OPERATOR  #24 
RANK:  2LT 

MOS:  14E 

PREFERENCE :  MONOCHROME 

REASONS:  The  colors  used  (pastels)  I  did  not  like.  The  color  used  for  the 
Sector  was  not  good,  I  think  It  should  have  been  a  darker  color 
therefore  the  Range  Rings,  defended  areas  etc.  would  have  shown-up 
better.  Tlie  colors  used  were  too  soft.  It  seons  to  me  when  people 
are  dealing  with  a  ccrabat  situation  It  helps  to  have  sane  dark 
colors  -  however,  the  red  for  host lies  was  effective.  The 
monochrcrae  display  I  liked  better  because  there  were  not  alot  of 
colors  to  distinguish  from  only  shapes-thls  seems  easier  for  me. 
CHANGES:  Yes,  I  would  change  the  color-mentioned  on  previous  page. 

SPECIFIC  CHANGES: 

FRIENDLY  TRACKS:  Darker  color 

LNIPS,  PIPS,  TRAILS:  Same 

MISSILES:  Same 

FEB A:  Darker  -  maroon 

SEARCH  SECTOR:  Darker  -  green 

TRACK  SECTOR:  Darker 

RANGE  RINGS:  Darker  -  navy  possibly 

MASKED  TERRAIN:  Light  color 

CRG:  Same 

BATTALION  FLAG:  Same 

CURSOR:  Same  -  keep  lighter  color 

NORTH  SYMBOL:  Same 

WEAPONS  CONTROL  VOLUME-HOLD:  Darker  color 
WEAPONS  CONTROL  VOLUME-FREE:  Darker  color 
WEAPONS  CONTROL  VOLUME-TIGHT:  Darker  color 
RESTRICTED  VOLUME:  Darker  color 
PROHIBITED  VOLUME:  Darker 
DEFENDED  AFffiA/POINTS :  Darker 
SAFE  PASSAGE  CORRIDOR:  Same 

BACKGROUND:  Definitely  a  darker  background,  possibly  black 
OTHER  TABS:  Color  could  be  used  for  TLL/RT. 


